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Part I 


© 


Introduction and Perspectives 


1 


An Introduction to the Second 
Edition of The Cambridge 
Handbook of Expertise and Expert 
Performance: Its Development, 
Organization, and Content 


K. Anders Ericsson 
Department of Psychology, Florida State University, Florida 


The study of expertise and expert performance reached a significant 
milestone in 2006 when its first handbook was published (Ericsson, 


Charness, Hoffman, & Feltovich, 2006). In the ten subsequent years, the 


handbook surpassed 10,000 copies sold, which is pretty impressive for a book 
of almost 1,000 pages. During this last decade there has been a dramatic 
increase in articles and books reporting on expertise and expert performance. 
There are several edited books written about particular domains of expertise, 


such as sports expertise (Baker & Farrow, 2015) and developing sports 


expertise (Farrow & Baker, 2013), entrepreneurial expertise (Sarasvathy, 


2008), and design expertise (Lawson & Dorst, 2009). Other books have taken 
more general perspectives on the structure of expertise and its acquisition 


(Montero, 2016), the social aspects of how expertise is evaluated and experts 


evaluated (Collins & Evans, 2007), and the relation between skill acquisition 
and expertise (Johnson & Proctor, 2016). General books on the topics of 
expertise and expert performance have been published, focusing on 


professional development (Ericsson, 2009), accelerating the development of 


expertise (Hoffman et al., 2014), as noted earlier, and expertise in 


professional decision making (Hoffman, 2007). Another sign of impact is the 


large number of popular books describing how insights from the study of 
expertise and expert performance can inform individuals on how to improve 
their performance. A few examples of such popular books are Colvin (2008), 
Coyle (2009), Ericsson and Pool (2016), Foer (2011), Gladwell (2008), and 
Marcus (2012). This new edition of the handbook will update the most active 


areas of research and provide an up-to-date summary of our knowledge about 
perspectives, approaches, and methods in the study of expertise and expert 
performance as well as updated assessments of the knowledge of expertise 
and expert performance in different domains of expertise. There is also a new 
section identifying similar mechanisms that mediate expertise and expert 
performance across different domains, as well as generalizable issues and 


theoretical frameworks. 


Expert, Expertise, and Expert Performance: 
Dictionary Definitions 


Encyclopedias describe an Expert as “one who is very skillful and well- 


informed in some special field” (Webster’s New World Dictionary, 1968, 


p. 168), or “someone widely recognized as a reliable source of knowledge, 
technique, or skill whose judgment is accorded authority and status by the 
public or his or her peers. Experts have prolonged or intense experience 
through practice and education in a particular field” (Wikipedia). Expertise 
then refers to the characteristics, skills, and knowledge that distinguish 
experts from novices and less experienced people. In some domains there are 
objective criteria for finding experts, who are consistently able to exhibit 
superior performance for representative tasks in a domain. For example, 
chess masters will almost always win chess games against recreational chess 
players in chess tournaments, medical specialists are far more likely to 
diagnose a disease correctly than advanced medical students, and 
professional musicians can perform pieces of music in a manner that is 
unattainable for less skilled musicians. These types of superior reproducible 
performances on representative tasks capture the essence of the respective 
domains, and authors have been encouraged to refer to them as Expert 
Performance in this and the original handbook. 

It has been known for some time that in some domains it is difficult for 
non-experts to identify experts, and consequently researchers rely on peer- 
nominations by professionals in the same domain. However, people 


recognized by their peers as experts do not always display superior 


performance on domain-related tasks. Sometimes they are no better than 
novices even on tasks that are central to the expertise, such as selecting 
stocks with superior future value, treatment of psychotherapy patients, and 
forecasts (Ericsson & Lehmann, 1996). There are several domains where 
experts disagree and make inconsistent recommendations for action, such as 
recommending selling versus buying the same stock. For example, expert 
auditors’ assessments have been found to differ more from each other than 
the assessments of less experienced auditors (Bédard, 1991). Furthermore, 
experts will sometimes acquire differences from novices and other people as 
a function of their repetitive routines, that is, as a consequence of their 
extended experience rather than a cause for their superior performance. For 
example, medical doctors’ handwriting is less legible than that of other health 


professionals (Lyons, Payne, McCabe, & Fielder, 1998). In sum, Shanteau 


(1988) suggested that “experts” may not need a proven record of performance 
and can adopt a particular image and project “outwards signs of extreme self- 
confidence” (p. 211) to get clients to listen to them and continue to offer 
advice after negative outcomes. After all, the experts are nearly always the 
best qualified to evaluate their own performance and explain the reasons for 
any deviant outcomes. 

When the proposal for the first edition of the handbook was originally 
prepared, the outline focused more narrowly on the structure and acquisition 
of highly superior (expert) performance in many different domains (Ericsson, 
1996, 2004). In response to the requests of the reviewers of that proposal, the 
final outline of the handbook covered a broader field that included research 
on the development of expertise and how highly experienced individuals 
accumulate knowledge in their respective domains and eventually become 


socially recognized experts and masters. Consequently, to reflect the scope of 


the handbook it was entitled The Cambridge Handbook of Expertise and 
Expert Performance. The first edition of the handbook thus included a 
multitude of conceptions of expertise, including perspectives from education, 
sociology, and computer science, along with the more numerous perspectives 
from psychology emphasizing basic abilities, knowledge, and acquired skills. 
In this second edition there is an even more committed effort to include new 
perspectives, such as the evolution of expertise over many millennia, the 
phenomenology of expertise, and even the concept of expertise in non-human 
animals, such as service dogs and dogs herding sheep. In this introductory 
chapter, I will briefly introduce some general issues and describe the structure 
and content of the handbook as it was approved by Cambridge University 


Press. 


Tracing the Development of Our Knowledge 
of Expertise and Expert Performance 


Since the beginning of Western civilization there has been particular interest 
in the superior knowledge that experts have acquired in their domain of 
expertise. The body of knowledge that experts accrue in their domain is a 
particularly important difference between experts and other individuals. 
Much of this knowledge can be verbally described and shared with others to 
benefit decision making in the domain and can help educate students and 
facilitate their progress toward expertise. The special status of the knowledge 
of experts in their domain of expertise is acknowledged even as far back as 


the Greek civilization. Socrates said that: 


I observe that when a decision has to be taken at the state assembly 
about some matter of building, they send for the builders to give their 
advice about the buildings, and when it concerns shipbuilding they send 
for the shipwrights, and similarly in every case where they are dealing 
with a subject which they think can be learned and taught. But if anyone 
else tries to give advice, who they don’t regard as an expert, no matter 
how handsome or wealthy or well-born he is, they still will have none of 
him, but jeer at him and create an uproar, until either the would-be 
speaker is shouted down and gives up of his own accord, or else the 


police drag him away or put him out on the order of the presidents. 


(Plato, 1991, pp. 11-12) 


Aristotle relied on his own senses as the primary source of scientific 


knowledge and sought out beekeepers, fishermen, hunters, and herdsmen to 


get the best and most reliable information for his books on science (Barnes, 
2000). He even tried to explain occasional incorrect reports from some of his 
informants about how offspring of animals were generated. For example, 
some of them suggested that “the ravens and the ibises unite at the mouth” 


(Aristotle, 1943, p. 315). But Aristotle notes: “It is odd, however, that our 


friends do not reason out how the semen manages to pass through the 
stomach and atrive in the uterus, in view of the fact that the stomach concocts 
everything that gets into it, as it does the nourishment” (pp. 315 & 317). 
Similarly, “those who assert that the female fishes conceive as a result of 
swallowing the male’s semen have failed to notice certain points” (p. 311). 
Aristotle explains that “Another point which helps to deceive these people is 
this. Fish of this sort take only a very short time over their copulation, with 
the result that many fishermen never even see it happening, for of course no 
fishermen ever watches this sort of thing for the sake of pure knowledge” (p. 
313). Much of Aristotle’s knowledge comes, at least partly, from consensus 
reports of professionals. 

Much later during the Middle Ages, craftsmen formed guilds to protect 
themselves from competition. Through arrangements with the mayor and/or 
monarch they obtained a monopoly on providing particular types of handcraft 


and services with set quality standards (Epstein, 1991). They passed on their 


special knowledge of how to produce products, such as lace, barrels, and 
shoes, to their students (apprentices). Apprentices would typically start at 
around age 14 and commit to serve and study with their master for around 
seven years — the length of time varied depending on the complexity of the 
craft and the age and prior experience of the apprentice (Epstein, 1991). Once 
an apprentice had served out their contract they were given a letter of 


recommendation and were free to work with other masters for pay, which 


often involved traveling to other cities and towns — they were therefore 
referred to as journeymen. When a journeyman had accumulated enough 
additional skill and saved enough money he, or occasionally she, would often 
return to his home town to inherit or purchase a shop with tools and apply to 
become a master of the guild. In most guilds they required inspection of the 
journeyman’s best work, i.e. master pieces, and in some guilds they 
administered special tests to assess the level of performance (Epstein, 1991). 
When people were accepted as masters they were held responsible for the 
quality of the products from their shop and were thereby allowed to take on 
the training of apprentices (see the chapter by Amirault & Branson, 2006, in 
the first edition of the handbook on the progression toward expertise and 
mastery of a domain). 

In a similar manner, the scholars’ guild was established in the twelfth 
and thirteenth centuries as “a univeristas magistribus et pupillorum,” or 
“suild of masters and students” (Krause, 1996, p. 9). Influenced by the 
University of Paris, most universities conducted all instruction in Latin, 
where the students were initially apprenticed as arts students until they 
successfully completed the preparatory (undergraduate) program and were 
admitted to the more advanced programs in medicine, law, or theology. To 
become a master, the advanced students needed to satisfy “a committee of 
examiners,” then publicly defend a thesis, often in the town square and with 


local grocers and shoemakers asking questions (Krause, 1996, p. 10). The 


goal of the universities was to accumulate and explain knowledge and in the 
process masters organized the existing knowledge (see Amirault & Branson, 
2006). With the new organization of existing knowledge of a domain, it was 
no longer necessary for individuals to discover the relevant knowledge and 


methods by themselves. 


Today’s experts can rapidly acquire the knowledge originally discovered 
and accumulated by preceding expert practitioners by enrolling in courses 
taught by skilled and knowledgeable teachers using specially prepared 
textbooks. For example, in the thirteenth century Roger Bacon argued that it 
would be impossible to master mathematics by the then known methods of 
learning (self-study) in less than 30 to 40 years (Singer, 1958). Today the 
roughly equivalent material (calculus) is taught in highly organized and 
accessible form in every high school. 

Sir Francis Bacon is generally viewed as one of the architects of the 
Enlightenment period of Western civilization and one of the main proponents 
of the benefits of generating new scientific knowledge. In 1620 he described 
in his book Novum Organum his proposal for collecting and organizing all 
existing knowledge to help our civilization engage in learning to develop a 
better world. In it, he appended a listing of all topics of knowledge to be 
included in Catalogus Historarium Particularium. It included a long list of 
skilled crafts, such as “History of weaving, and of ancillary skills associated 
with it,” “History of dyeing,” “History of leather-working, tanning, and of 
associated ancillary skills” (Rees & Wakely, 2004, p. 483). 

The guilds guarded their knowledge and their monopoly of production. 


It is therefore not surprising that the same forces that eventually resulted in 
the French Revolution were not directed only at the oppression by the king 
and the nobility, but also against the monopoly of services provided by the 
members of the guilds. Influenced by Sir Francis Bacon’s call for an 
encyclopedic compilation of human knowledge, Diderot and D’Alembert 
worked on assembling all available knowledge in the first Encyclopédie 
(Diderot & D’ Alembert, 1966—67), which was published in 1751-80. 


Diderot was committed to the creation of comprehensive descriptions of 


the mechanical arts to make their knowledge available to the public and 
encourage research and development in all stages of production and all types 
of skills, such as tanning, carpentry, glassmaking, and ironworking 


(Pannabecker, 1994) along with descriptions of how to sharpen a feather for 


writing with ink as shown in Figure 1.1. His goal was to describe all the raw 
materials and tools that were necessary, along with the methods of 
production. Diderot and his associate contributors had considerable 
difficulties gaining access to all the information because of the unwillingness 
of the guild members to answer their questions. Diderot even considered 
sending some of his assistants to become apprentices in the respective skills 
to gain access to all the relevant information (Pannabecker, 1994). In spite of 
all the information and pictures (diagrams of tools, workspaces, procedures, 
etc. as illustrated in Figure 1.2 showing one of several plates of the process of 
printing) provided in the Encyclopédie, Diderot was under no illusion that the 
provided information would by itself allow anyone to become a craftsman in 
any of the described arts and wrote: “It is handicraft that makes the artist, and 
it is not in Books that one can learn to manipulate” (Pannabecker, 1994, 
p. 52). In fact, Diderot did not even address the higher levels of cognitive 
activity, “such as intuitive knowledge, experimentation, perceptual skills, 
problem-solving, or the analysis of conflicting or alternative technical 


approaches” (Pannabecker, 1994, p. 52). 
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Figure 1.1 An illustration of how to sharpen a goose feather for writing 
with ink from Plate IV in the entry on “Ecriture” in the 23rd volume of 
Encyclopédie ou dictionnare de raisonné des sciences, des arts et des 
métiers (Diderot & D’Alembert, 1966-67). 
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Figure 1.2 An illustration of the workspace of a printer with some of his 
type elements from Plate I in the entry on “Imprimerie” in the 28th volume 
of Encyclopédie ou dictionnare de raisonné des sciences, des arts et des 
métiers (Diderot & D’ Alembert, 1966—67). 


A couple of years after the French Revolution the monopoly of the 


guilds was eliminated (Fitzsimmons, 2003) including the restrictions on the 


practice of medicine and law. After the American Revolution and the creation 


of the United States of America laws were initially created to require that 
doctors and lawyers be highly trained based on the apprenticeship model, but 
pressure to eliminate elitist tendencies led to the repeal of those laws. From 
1840 to the end of the nineteenth century there was no requirement for 
certification to practice medicine and law in the USA (Krause, 1996). 
However, with time both France and the USA realized the need to restrict 
vital medical and legal services to qualified professionals and developed 
procedures for training and certification. 

Over the last couple of centuries there have been several major changes 
in the relation between master and apprentice. For example, before the 
middle of the nineteenth century children of poor families would often be 
taken on by teachers in exchange for a contractual claim for part of the future 
dancers’, singers’, or musicians’ earnings as an adult (Rosselli, 1991). Since 
then the state has gotten more involved in the training of their expert 
performers, even outside the traditional areas of academia and professional 
training in medicine, law, business, and engineering. In the late nineteenth 
century public institutions, such as the Royal Academy of Music, were 
established to promote the development of very high levels of skill in music 
to allow native students to compete with better trained immigrants (Rohr, 
2001). In a similar manner during the latter part of the twentieth century, 
many countries invested in schools and academies for the development of 
highly skilled athletes for improved success in competitions during the 
Olympic Games and World Championships (Bloomfield, 2004). 

More generally, over the last century there have been economic 
developments with public broadcasts of competitions and performances that 
generate sufficient revenue for a number of domains of expertise, such as 


sports and chess, to support professional full-time performers as well as 


coaches, trainers, and teachers. In these new domains, along with the 
traditional professions, current and past expert performers continue to be the 
primary teachers at advanced levels (masters) and their professional 
associations have the responsibility for certifying acceptable performance and 
the permission to practice. Thus they have the clout to influence training in 
professional schools, such as law, medical, nursing, and business schools — 
“testing is the tail that wags the dog” (Feltovich, personal communication). 
The accumulation of knowledge about the structure and acquisition of 
expertise in a given domain as well as knowledge about instruction and 
training of future professionals have, until quite recently, occurred almost 
exclusively within each domain, with little cross-fertilization of domains in 
terms of teaching, learning methods, and skill training techniques. 

It is not immediately apparent what is generalizable across such diverse 
domains of expertise as music, sport, medicine, and chess. What could 
possibly be shared by the skills of playing difficult pieces by Chopin, running 
a mile in less than four minutes, and playing chess at a high level? The 
premise for a field studying expertise and expert performance is that there are 
sufficient similarities in the theoretical principles mediating the phenomena 
and the methods for studying them in different domains that it would be 
possible to propose a general theory of expertise and expert performance. All 
of these domains of expertise have been created by humans and thus the 
accumulated knowledge and skills are likely to reflect similarities in structure 
reflecting human biological and psychological factors as well as cultural 
factors. This raises many challenging problems for methodologies seeking to 
describe the organization of knowledge and to identify the mechanisms 
mediating expert performance that generalize across domains. 


Once we know how experts organize their knowledge and _ their 


performance, is it possible to improve the efficiency of learning to reach 
higher levels of expert performance in these domains? It should also be 
possible to determine why different individuals improve their performance at 
different rates and why different people reach very different levels of final 
achievement. Would a deeper understanding of the development and its 
mediating mechanisms make it possible to select individuals with unusual 
potential and to design better developmental environments to increase the 
proportion of performers who reach the highest levels? Would it even be 
possible to facilitate the development of those rare individuals who make 


major creative contributions to their respective domains? 


Conceptions of Generalizable Aspects of 
Expertise 


Several different theoretical frameworks have focused on broad issues on 
attaining expert performance that generalize across different domains of 


expertise. 


Individual Differences in Mental Capacities 


A widely accepted theoretical concept argues that general innate mental 
capacities mediate the attainment of exceptional performance in most 
domains of expertise. In his famous book, Hereditary Genius, Galton 
(1869/1979) proposed that across a wide range of domains of intellectual 
activity the same innate factors are required to attain outstanding achievement 
and designation as a genius. He analyzed eminent individuals in many 
domains in Great Britain and found that these eminent individuals were very 
often the offspring of a small number of families — with much higher 
frequency than could be expected by chance. The descendants from these 
families were much more likely to make eminent contributions in very 
diverse domains of activity, such as becoming famous politicians, scientists, 
judges, musicians, painters, and authors. This observation led Galton to 
suggest that there must be a heritable potential that allows some people to 
reach an exceptional level in any one of many different domains. After 
reviewing the evidence that height and body size were heritable Galton 
(1869/1979) argued: “Now, if this be the case with stature, then it will be true 
as regards every other physical feature — as circumference of head, size of 
brain, weight of gray matter, number of brain fibers, &c.; and thence, by a 
step on which no physiologist will hesitate, as regards mental capacity” (pp. 
31-32, emphasis added). 

Galton clearly acknowledged the need for training to reach high levels 
of performance in any domain. However, he argued that improvements are 
rapid only in the beginning of training and that subsequent increases become 


increasingly smaller, until “maximal performance becomes a rigidly 


determinate quantity” (p. 15). Galton developed a number of different mental 
tests of individual differences in mental capacity. Although he never related 
these measures to objective performance of experts on particular real-world 
tasks, his views led to the common practice of using psychometric tests for 
admitting students into professional schools and academies for arts and sports 
with severely limited availability of slots. These tests of basic ability and 
talent were believed to identify the students with the capacity for reaching the 
highest levels. 

In the twentieth century scientists began testing large groups of experts 
to measure their powers of mental speed, memory, and intelligence with 
psychometric tests. When the experts’ performances were compared to 
control groups of comparable education there was no evidence supporting 
Galton’s hypothesis of a general superiority for experts, because the 
demonstrated superiority of experts was found to be specific to certain 
aspects related to the particular domain of expertise. For example, the 
superiority of the chess expert’s memory was constrained to regular chess 
positions and did not generalize to other types of materials (Djakow, 
Petrowski, & Rudik, 1927). Not even IQ could distinguish the best among 
chess players (Doll & Mayr, 1987) nor the most successful and creative 
among artists and scientists (Taylor, 1975). 

In an article in the Annual Review of Psychology, Ericsson and Lehmann 
(1996) found that (1) measures of basic mental capacities are not valid 
predictors of attainment of expert performance in a domain, (2) the superior 
performance of experts is often very domain specific and transfer outside 
their narrow area of expertise is surprisingly limited, and (3) systematic 
differences between experts and less proficient individuals nearly always 


reflect attributes acquired by the experts during their lengthy training. Since 


the first edition of this book there have been several special issues directed to 
the discussion of various factors influencing the development of expert 
performance, such as a special issue of the International Journal of Sport 
Psychology on “Nature, Nurture, and Sport Performance” (Baker & Davids, 
2007), a special issue of High Ability Studies on “Expertise and Giftedness 


Research” (Stoeger, 2007), and a special issue of Intelligence on “Acquiring 


Expertise: Ability, Practice, and Other Influences” (Detterman, 2014). 


Expertise as the Extrapolation of Everyday Skill to Extended 
Experience 
A second general type of theoretical framework is based on the assumption 
that the same learning mechanisms that account for the acquisition of 
everyday skills can be extended to the acquisition of higher levels of skills 
and expertise. Studies in the nineteenth century proposed that the acquisition 
of high levels of skills was a natural consequence of extended experience in 
the domains of expertise. For example, Bryan and Harter (1899) argued that 
ten years of experience were required to become a professional telegrapher. 
The most influential and pioneering work on expertise was conducted in the 


1940s by Adrian de Groot (1978), who invited international chess masters 


and skilled club players to “think aloud” while they selected the best move 
for chess positions. His analyses of the protocols showed that the elite players 
were able to recognize and generate chess moves that were superior to skilled 
club players by relying on acquired patterns and planning. De Groot’s 
dissertation was later translated into English in the late 1960s and early 1970s 


(de Groot, 1978) and had substantial impact on the seminal theory of 


expertise proposed by Herb Simon and Bill Chase (Simon & Chase, 1973). 


In the 1950s and 1960s Newell and Simon proposed how information 
processing models of human problem-solving could be implemented as 
computer programs, such as the General Problem Solver (Ernst & Newell, 
1969). In their seminal book, Human Problem Solving, Newell and Simon 
(1972) argued that domain-general problem-solving was limited and that 
thinking involved in solving most tasks could be represented as the execution 


of a sequence of production rules, such as IF <pattern>, THEN <action> that 


incorporated specific knowledge about the task environment. In their theory 
of expertise, Simon and Chase (1973) made the fundamental assumption that 
the same patterns (chunks) that allowed the experts to retrieve suitable 
actions from memory were the same patterns that mediated experts’ superior 
memory for the current situation in a game. 

Chase and Simon (1973) redirected the focus of research toward 
studying performance on memory tasks as a more direct method of studying 
the characteristics of patterns that mediate improvement in skill. They found 
that there was a clear relation between the number of chess pieces recalled 
from briefly presented chess positions and the player’s level of chess 
expertise. Grand masters were almost able to reproduce entire chessboards 
(24—26 pieces) by recalling a small number of complex chunks whereas 
novices could only recall around four pieces, where each piece was a chunk. 
The masters’ superior memory was assumed to depend on an acquired body 
of many different patterns in memory, because their memory for randomly 
rearranged chess configurations was markedly reduced. In fact, they could 
only recall around five to seven pieces, which was only slightly better than 
the recall of novices. 

Experts’ superiority for representative but not randomly rearranged 
stimuli has since been demonstrated in a large number of domains. The 
relation between the mechanisms mediating memory performance and the 
ones mediating representative performance in the same domains has been 
found to be much more complex than originally proposed by Simon and 
Chase (1973) (for a more up-to-date review on the topic of experts’ superior 
memory for representative information in their domain see Ericsson, Chapter 


36, and Gobet & Charness, Chapter 31, this volume). 


Expertise as Qualitatively Different Representation and 
Organization of Knowledge 
A different family of approaches drawing on the Simon—Chase theory of 
expertise has focused on the content and organization of the experts’ 
knowledge (Chi, Feltovich, & Glaser, 1981; Chi, Glaser, & Rees, 1982) and 


on methods to extract the experts’ knowledge to build computer-based 


models emulating the experts’ performance (Hoffman, 1992). These 
approaches have studied experts, namely individuals who are socially 
recognized as experts and/or distinguished by their extensive experience 
(typically over ten years) and by knowledge of a particular subject attained 
through instruction, study, or practical experience. The work of Micheline 
Chi, Paul Feltovich, and Robert Glaser (1981) examined the representations 
of knowledge and problem solutions in academic domains, such as physics. 
Of particular importance, Chi (1981) studied children with extensive 
knowledge of chess and dinosaurs and found these children displayed many 
of the same characteristics of the knowledge representations of adult experts. 
This work on expertise is summarized in this volume by Feltovich, Prietula, 
and Ericsson, Chapter 6, and Lintern, Moon, Klein, and Hoffman, Chapter 
i. 

In a parallel development in computer science of the late 1970s and 
early 1980s, Edward Feigenbaum and other researchers in the area of 
artificial intelligence and cognitive science attempted to elicit the knowledge 


of experts (Hoffman, 1992) and to incorporate their knowledge in computer 


models (expert systems) that sought to replicate some of the decision making 


and behavior of experts (see in this volume Buchanan, Davis, Smith, & 


Feigenbaum, Chapter 7, and Lintern et al., Chapter 11). There has been a 
long-standing controversy over whether highly experienced experts are 
capable of articulating the knowledge and methods that control their 
generation of appropriate actions in complex situations. 

The tradition of skill acquisition of Bryan and Harter (1899), Fitts and 
Posner (1967), and Simon and Chase (1973) assumed that expert 
performance was associated with automation and virtually effortless 
performance based on pattern recognition and direct access of actions. 
However, Polanyi (1962, 1966) is generally recognized as the first critic who 
saw that non-conscious and intuitive mediation limits the possibility of 
eliciting and mapping the knowledge and rules that mediated experts’ 
intuitive actions. Subsequent discussion of the development of expertise by 
Dreyfus and Dreyfus (1986) and Benner (1984) has argued that the highest 
levels of expertise are characterized by contextually based intuitive actions 
that are difficult or impossible to report verbally. Several chapters in this 
handbook propose methods for uncovering tacit knowledge about successful 
development of expertise (Cianciolo & Sternberg, Chapter 39), about 
methods of work through observation (Clancey, 2006), critical incident 
reports, concept maps, and decision ladders (Lintern et al., Chapter 11), 
reaction times to perceptual stimuli (Landy, Chapter 10), superior 
anticipation of opponents’ action (Abermethy, Farrow, & Mann, Chapter 35), 
and traditional psychometric analyses of individual differences in traits 
related to attained performance (Ackerman & Beier, Chapter 13). Other 
investigators have collected concurrent verbal reports (think-aloud protocols) 
to monitor the experts’ performance while they respond to representative 
situations from their domain (Ericsson, Chapter 12). These verbalized 


thoughts have raised issues about how experts have acquired memory skills 


to allow them to maintain efficient access to diverse information relevant to 
the generation of performance (long-term working memory, Ericsson, 
Chapter 36, and situational awareness, Endsley, Chapter 37). This latter 
evidence on expertise suggests that expert performers have to actively retain 
and refine their mental representations for monitoring and controlling their 


performance. 


Expertise as Elite Achievement Resulting from Superior Learning 
Environments 
There are other approaches to the study of expertise that have focused on 
objective achievement. There is a long tradition of influential studies with 


interviews of peer-nominated eminent scientists (Roe, 1952) and analyses of 


biographical data on Nobel Prize winners (Zuckerman, 1977) (see Simonton, 
1994, for a more extensive account). In a seminal study, Benjamin Bloom 


and his colleagues (Bloom, 1985a) interviewed international-level performers 


from six different domains of expertise ranging from swimming to molecular 
genetics. All of the 120 participants had won prizes at international 
competitions in their respective domains. They were all interviewed about 
their development, along with their parents, teachers, and coaches. For 
example, Bloom and his colleagues collected information on the development 
of athletes who had won international competitions in swimming and tennis. 
They also interviewed artists who had won international competitions in 
sculpting and piano playing and scientists who had won international awards 
in mathematics and molecular biology. In each of these six domains Bloom 
(1985b) found evidence for uniformly favorable learning environments for 
the participants in all the domains. Bloom (1985b) concluded that the 
availability of early instruction and support by their families appeared to be 
necessary for attaining an international level of performance as an adult. He 
found that the elite performers typically started early to engage in relevant 
training activities in the domain and were supported both by exceptional 
teachers and by committed parents. These topics are covered in this 


handbook through up-to-date reviews of historiometric approaches to the 


development of professional excellence (Simonton, Chapter 18) and of case 
studies of experts (Mumford, McIntosh, & Mulhearn, Chapter 17). In a new 
addition to the handbook Elferink-Gemser, te Wierike, and Visscher (Chapter 


16) review longitudinal studies of large groups of expert performers. 


Expertise as Reliably Superior (Expert) Performance on 
Representative Tasks 
It is difficult to identify the many mediating factors that might have been 
responsible for an elite performer to win an award and to write a 
groundbreaking book. When eminence and expertise are based on a singular 
or small number of unique creative products, such as books, paintings, or 
music compositions, it is rarely possible to identify and scientifically study 
the key factors that allowed these people to produce these achievements. 
Consequently, Ericsson and Smith (1991) proposed that the study of expertise 
with laboratory rigor requires representative tasks that capture the essence of 
expert performance in a specific domain of expertise. For example, a world- 
class sprinter will be able to reproduce superior running performance on 
many tracks and even indoors in a large laboratory. Similarly, de Groot 
(1978) found that the ability to select the best move for presented chess 
positions is the best correlate of chess ratings and performance at chess 
tournaments — a finding that was replicated (van der Maas & Wagenmakers, 
2005). Once it is possible to reproduce the reliably superior performance of 
experts in a controlled setting, such as a laboratory, it then becomes feasible 
to examine the specific mediating mechanisms with experiments and process- 
tracing techniques, such as think-aloud verbal reports (see Ericsson, Chapter 


12, in this volume, and Ericsson & Smith, 1991). The discovery of 


representative tasks that measure adult expert performance under 
standardized conditions in a controlled setting, such as a laboratory, makes it 
possible to measure and compare the performance of less skilled individuals 


on the same tasks. Even more importantly, it allows scientists to test aspiring 


performers many times during their development of expertise, allowing the 
measurement of gradual increases in performance. 

The new focus on measurement of expert performance with standardized 
tasks revealed that “experts,” i.e. individuals identified by their reputation or 
their extensive experience, are not always able to exhibit reliably superior 
performance. There are at least some domains where “experts” perform no 
better than less trained individuals and that sometimes experts’ decisions are 
no more accurate than beginners’ decisions and simple decision aids (Bolger 
& Wright, 1992; Camerer & Johnson, 1991). Most individuals who start as 


active professionals or as beginners in a domain change their behavior and 


increase their performance for a limited time until they reach an acceptable 
level. Beyond this point, however, further improvements appear to be 
unpredictable and the number of years of work and leisure experience in a 
domain is a poor predictor of attained performance (Ericsson & Lehmann, 
1996). Hence, continued improvements (changes) in achievement are not 
automatic consequences of more experience and, in those domains where 
performance consistently increases, aspiring experts seek out particular kinds 
of training tasks designed for the particular performers by their teachers and 
coaches (deliberate practice) (Ericsson, Krampe, & Tesch-R6mer, 1993). 
Several chapters in this revised handbook describe how deliberate practice 
can change the mechanisms mediating the experts’ superior performance and 
that the accumulated amounts of deliberate practice are related to attained 
level of performance (see Ericsson, Chapter 38). Baker, Hodges, and Wilson 
(Chapter 15) review methods for collecting information about practice 


activities using concurrent and retrospective methods. 


General Comments 


In summary, there are a broad range of approaches to the study of the 
structure and acquisition of expertise as well as expert performance. 
Although individual researchers and editors may be primarily pursuing one of 
the approaches, this handbook has been designed to cover a wide range of 
different approaches and research topics in order to allow authors to describe 
their own views. However, the authors have been encouraged to describe 
explicitly their empirical criteria for their key concepts, such as expertise, 
experts, and expert performance. For example, the authors have been asked to 
report if the cited research findings involve experts identified by social 
criteria, criteria of lengthy domain-related experience, or criteria based on 
reproducibly superior performance on a particular set of tasks representative 


of the individuals’ domain of expertise. 


General Outline of the Handbook 


The second edition of this handbook is organized into seven general sections. 
First, Part I introduces the handbook with brief accounts of general 
perspectives on expertise. In addition to this introductory chapter that outlines 
the organization of the handbook, there are four chapters. All of the four 
chapters are important new additions to the handbook. Collins and Evans 
(Chapter 2) give a sociological perspective of expertise based on 
philosophical analysis, Dall’Alba (Chapter 3) describes expertise from a 
phenomenological perspective based on the concept of the lifeworld, and 
Winegard, Winegard, and Geary (Chapter 4) take an evolutionary perspective 
on expertise and distinguish natural expertise, such as hunting, from non- 
functional expertise, such as chess. Finally Helton and Helton (Chapter 5) 
describe expertise displayed by non-humans, such as trained dogs detecting 
illegal drugs or herding livestock. 

Part IJ of the revised handbook contains reviews of the historical 
development of the study of expertise from the perspective of different 
disciplines. Feltovich, Prietula, and Ericsson (Chapter 6) review the recurrent 
themes in the study of expertise from a psychological perspective. Buchanan, 
Davis, Smith, and Feigenbaum (Chapter 7) trace the historical development 
of using computers to model expertise, especially in the form of expert and 
knowledge-based systems. Billett, Harteis, and Gruber (Chapter 8) describe 
occupational expertise and its development based on experiences in the 
workplace. Finally, Mieg and Evetts (Chapter 9) describe the historical 


development of professionals and experts from a social perspective. 


The next two sections of the handbook review the core methods for 
studying the structure (Part III) and acquisition (Part IV) of expertise and 
expert performance. Part II] focuses on how expertise and expert 
performance can be explained by observable differences between experts and 
novices. In the first chapter in this section David Landy (Chapter 10) 
describes how the development of expertise can influence even processes of 
perception. Lintern et al. (Chapter 11) describe how the knowledge of experts 
has been elicited using the critical incident method, concept maps, and 
decision ladders. Ericsson (Chapter 12) describes the method of protocol 
analysis, which involves eliciting and recording the thought processes of 
experts when they respond to representative tasks from their domain of 
expertise. Ackerman and Beier (Chapter 13) describe psychometric 
approaches to expertise and identifying traits (cognitive, affective, and 
conative) that predict individual differences in its development. Finally 
Bilali¢ and Campitelli (Chapter 14) review methods to study changes in the 
neural structure and pattern of activation of the brain associated with 
expertise. 

Part IV contains chapters examining methods for studying how skill, 
expertise, and expert performance develop and their relation to practice and 
other types of activities during the development. In the first chapter, Baker et 
al. (Chapter 15) describe methods and findings related to concurrent and 
retrospective assessment of these activities to performance. Elferink-Gemser 
et al. (Chapter 16) review the methodology and findings from longitudinal 
studies of groups of individuals developing achievement and performance. 
Mumford et al. (Chapter 17) describe how the case method for studying 
individuals’ development can inform about the acquisition of expertise and 


expert performance. In the final chapter of this section, Dean Simonton 


(Chapter 18) reviews the methods of historiometrics and how data about the 
development of eminent performers can be collected and analyzed. 

Part V consists of 16 chapters that provide up-to-date reviews of our 
current knowledge about expertise and expert performance in particular 
domains and represents the core of this handbook. The chapters in Part V 
have been broken down into two subsections. Part V.I is focused on different 
types of professional expertise. In the first chapter Norman, Grierson, 
Sherbino, Hamstra, Schmidt, and Mamede (Chapter 19) review our rapidly 
expanding knowledge about expertise in medicine and surgery as well as new 
training methods including simulators. Durso, Dattel, and Pop (Chapter 20) 
review the new research on expertise in transportation, especially driving and 
the effect of experience and training on hazard perception. In a completely 
new addition to the handbook Cross (Chapter 21) describes the new emerging 
domain of expertise in design based on studies with interviews and protocol 
analysis. In another new addition Dew, Ramesh, Read, and Sarasvathy 
(Chapter 22) review the knowledge of expertise among entrepreneurs and 
focus on the skill of requesting resources for new projects (The Ask). Kellogg 
(Chapter 23) has updated and expanded his review of expertise among 
professional writers and emphasizes the importance of other factors than 
writing ability such as knowledge of the topic and accessible memory for the 
already generated text. In a new addition Stigler and Miller (Chapter 24) 
review the societally important topic of expertise among teachers and identify 
the “pseudo expertise in teaching” as an obstacle to progress and outline how 
teachers can be helped to become more effective in improving their students’ 
performance. Mosier, Fischer, Hoffman, and Klein (Chapter 25) describe the 
Naturalistic Decision Making approach to the examination and training of 


expert decision making in complex dynamic situations in everyday life. In a 


new addition to the handbook Cokely, Feltz, Ghazal, Allan, Petrova, and 
Garcia-Retamero (Chapter 26) review evidence on general decision making 
abilities that generalize across everyday contexts, finding that superior 
decision performance among both experts and non-experts primarily results 
from acquired specialized knowledge and probabilistic inductive reasoning 
skills (statistical numeracy and risk literacy). In the last chapter of Part V.I 
Sonesh, Lacerenza, Marlow, and Salas (Chapter 27) review the emerging 
evidence on how expert teams are more than the sum of all team members’ 
expertise and emphasize the importance of the teams’ adaptability, shared 
cognition, and leadership. 

Part V.I] contains chapters that review expert performance in the more 
traditional domains of games, such as chess, the arts, such as music, and 
sports. In the first chapter of the subsection Lehmann, Gruber, and Kopiez 
(Chapter 28) provide an updated review on the development of expert 
performance in music and its relation to the age of starting practice and the 
quality/quantity of different types of practice. Altenmiiller and Furuya 
(Chapter 29) review evidence for the view that favorable adaptations of the 
brain are associated with superior performance and how maladaptive changes 
of the brain due to overtraining can account for the inability to control music 
playing, such as violinist’s cramp. For the domains of drawing and painting 
Kozbelt and Ostrofsky (Chapter 30) examine the evidence for differences in 
general and specific perceptual and motor processes as a function of level of 
artistic skill. The classic domain of chess expertise is reviewed by Gobet and 
Charness (Chapter 31), who examine factors associated with individual 
differences in the ability to select superior chess moves, such as age of 
starting practice and amount of accumulated practice. Butterworth (Chapter 


32) describes the evidence primarily on the development of expertise in 


mathematical calculation and discusses the effects of mental ability (natural 
ability), motivation (zeal), and practice (hard work). In a new addition to the 
handbook Macis, Garnier, Vilkaiteé, and Schmitt (Chapter 33) review 
evidence on expertise in a foreign language by examining the development of 
learning and mastery of the critical vocabulary. In the final chapter of Part 
V.II Williams, Ford, Hodges, and Ward (Chapter 34) review expertise in 
sports and focus on the specificity and adaptability of expert athletes. 

Part VI of the handbook is a new addition and addresses an important 
issue in the study of expertise and expert performance. In spite of the 
specificity of superior performance in a given domain, is it possible to 
identify mechanisms mediating performance in different domains which 
reveal a similar abstract structure? In the first chapter of this section 
Abernethy et al. (Chapter 35) show how the superior speed of reacting by 
experts compared to less skilled individuals can be accounted for by earlier 
anticipation of opponents’ actions and better control and organization of their 
acquired motor processes. Ericsson (Chapter 36) shows how experts develop 
skills to maintain rapid access to information relevant to their current 
situations (long-term working memory). In the last chapter of Part VI 
Endsley (Chapter 37) reviews the research on experts’ superior mental 
models based on perception, comprehension of the current situation, and 
prediction of future situations (situation awareness). 

In Part VII the focus is on general theoretical issues that cut across 
different domains of expertise to provide reviews of the current state of 
knowledge. The first chapter, by Ericsson (Chapter 38), reviews the effects 
on attained performance from engagement in different types of domain- 
related activities, such as playing games, professional experience, solitary 


practice, and deliberate practice led by a teacher or coach. Cianciolo and 


Sternberg (Chapter 39) provide an updated review of the relation between 
expertise and central concepts/frameworks, such as practical intelligence, 
tacit knowledge, and related ecological theories. In a new addition Kalyuga 
and Sweller (Chapter 40) describe how instructional supports reduce 
cognitive load and improve learning for novice learners, but the same 
supports reduce the rate of further learning by more knowledgeable 
individuals and experts (the expertise reversal effect). Weisberg (Chapter 41) 
discusses the mechanisms mediating creative advances and shows how the 
expertise view provides superior accounts of the source of creativity. In the 
last chapter of the handbook, Krampe and Charness (Chapter 42) review the 
effects of aging deficits on tests of general cognitive ability for older 
participants. They find that these types of reduced performance on tests do 
not inevitably lead to reduced performance of experts, who are able to 
counteract reduction in the performance effects of aging with goal-directed 


practice. 


Conclusion 


This second edition of the handbook was designed to provide researchers, 
students, teachers, coaches, and anyone interested in attaining expertise with 
an up-to-date comprehensive reference to methods, findings, mechanisms, 
and theories related to expertise and expert performance. It is designed to be 
an essential tool for researchers, professionals, and students involved in the 
study or the training of expert performance and a necessary source for college 
and university libraries as well as public libraries. In addition, the volume is 
designed to provide a suitable text for graduate courses on expertise and 
expert performance. More generally, it is likely that professionals, graduate 
students, and even undergraduates who aspire to higher levels of performance 
in a given field can learn from experts’ pathways to superior performance in 
similar domains. 

Many researchers studying expertise and expert performance are excited 
and personally curious about the established research finding that most types 
of traditional expertise in competitive activities require years and decades of 
extended efforts to improve in order to acquire the mechanisms mediating 
world-class performance. There is considerable knowledge that is 
accumulating across many domains about the acquisition and refinement of 
these mechanisms during an extended period of training and practice. The 
generalizable insights range from the characteristics of ideal training 
environments with teachers and coaches, to the methods for fostering 


motivation by providing both emotional support and attainable training tasks 


of a suitable difficulty level. This theoretical framework has several 
implications. 

It implies that if someone is interested in the upper limits of human 
performance, and the most effective training to achieve the highest attainable 
levels, they should study the training techniques and performance limits of 
experts who have spent their entire life striving to maximize their 
performance in a particular domain. This assumption also implies that the 
study of expert performance will provide us with the best current evidence on 
what is humanly possible to change and improve with today’s methods of 
training and how these elite performers are able to achieve their highest levels 
of performance. Given that performance levels are increasing over decades 
and centuries in most domains of expertise, scientists will need to work with 
elite performers and their coaches to discover jointly the ever-increasing 
levels of improved performance. 

The framework has implications for education and professional training 
of performance for all the introductory levels that eventually lead up to the 
expert levels in professional domains of expertise. By examining how the 
prospective expert performers attained their initial beginning levels of 
achievement, we should be able to develop practice environments and foster 
learning methods that help people to attain the fundamental representations of 
the tasks and the self-regulatory skills that are necessary for the prospective 
experts to advance their learning to higher levels. 

With the rapid changes in the relevant knowledge and techniques 
required for most jobs nearly everyone will have to be capable of continuing 
their learning and even intermittently relearn aspects of their professional 
skills. The lifelong quest for improved adaptation to task demands will not be 


restricted to experts anymore. In order to be productive members of society 


we will all be encouraged to adopt the characteristics and the methods of the 
expert performers who continuously strive to attain and maintain their best 


level of achievement. 
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The sociological literature makes a distinction between expertise as a 
“performance” and expertise as a “property.” We begin with a summary of 
the relational or performative understanding of expertise but the larger part of 
the chapter treats expertise as a capacity or property acquired by socialization 
within the relevant social group. This alternative approach supports a detailed 
typology of expertise and touches on philosophical debates including the 
relationship between ideas and actions, the nature of tacit and explicit 
knowledge, and the role of the body in the acquisition of expertise. Focusing 


on socialization as the foundation of expertise also leads to the identification 


of a new kind of expertise — interactional expertise — that is acquired through 
linguistic socialization. We explain this idea in some detail, showing how 
interactional expertise makes language more central to the understanding of 
practical matters than practice itself. We conclude by summarizing the 


approach in a three-dimensional model of expertise. 


Expertise as Performance 


Treating expertise as a something that is “performed” means treating it as a 
relational or network phenomenon (Eyal, 2013) that is produced through its 
enactment in social settings. In this view, expertise is a status that individuals 
and organizations struggle to have attributed to them by others and which 
they must work to retain (Hilgartner, 2000). This, in turn, links expertise to 
issues of power and control with Foucault providing the classic exposition of 
“power/knowledge” (e.g. Foucault, 1978). Carr (2010) classifies sociological 


and anthropological work in this tradition into four main areas: 


e Apprenticeship, training, and socialization, where the aim is to 
describe the methods by which novices are initiated into a domain of 
skilled practice and the boundaries that define the group are 


maintained. In some cases (e.g. Lave & Wenger, 1991) this approach 


can serve to bolster the value of more traditional forms of learning; in 


others (e.g. Gieryn, 1999) it can be used to show the socially 


constructed, and hence contestable, nature of expert authority. 


e Authentication and evaluation, where the focus is on explicating the 
ways in which expert status, and hence the distinction between expert 
and non-expert, is enacted and legitimated in social settings. For 
example, researchers working within the ethnomethodological 
tradition of sociology might focus on the minutiae of interactions in 
order to reveal how the structure and topics of utterances, combined 
with non-verbal behaviors, are used to project authority and claim 
expert status (e.g. Lynch & Cole, 2005; Matoesian, 1999, 2008). Such 


work typically makes no claim to judge whether or not participants’ 
attributions of expertise are justified by any independent criteria. 
Instead, the point is to reveal the methods social actors use to 


construct and contest expert status (Coopmans & Button, 2014). 


Institutions and authorization, where the emphasis is on how expert 
knowledge is stabilized in both formal institutions and everyday 
practices. This might include studies of professions and the autonomy 
they are able to claim (e.g. Abbott, 1988; Turner, 2001), the 


difficulties of collaboration between specialists as exemplified by 


scientific research (e.g. Fisher et al., 2015; Galison, 1997), or the 
incorporation of science and technology into everyday practices like 


waste management (e.g. Woolgar & Neyland, 2013). 


Naturalization, where research uses the social constructivist 
approach to highlight the cultural and political assumptions that are 
embedded within dominant forms of expertise but which are so taken 
for granted that they are rarely questioned. Again, the field of science 
studies provides a good example of this kind of work with typical 
topics being the way problems are framed and what is allowed to 
count as evidence in situations of conflict (e.g. Delborne, 2008; Irwin, 
1995; Shapin, 2007; Wynne, 1992). 


Epistemic Injustice and the Limits of Attribution 


The network or relational model of expertise is normative insofar as it aims to 
reveal the contingent ways in which “particular claims and attributions of 
expertise come into being and are sustained and what the implications are for 


truth and justice” (Jasanoff, 2003, p. 398). The analysis typically documents 


how different social groups mobilize cultural and epistemic resources to 
challenge established ways of defining problems and is often framed as a 
critique of elite, top-down, or technocratic decision making (Callon, 1986; 


Fischer, 2009; Turner, 2003). One particularly striking outcome of this work 


has been to show that even the most privileged networks of expertise can be 
effectively challenged by organized groups of lay citizens and stakeholders 
(e.g. Epstein, 1996; Ottinger, 2010). 


Epistemic and political issues are often considered alongside each other 


in these studies. The same boundary work (Gieryn, 1999) that discredits 


local, experiential, or traditional knowledge also reinforces the social and 
technological outcomes that put indigenous or minority communities at 
increased risk, with Love Canal (Mazur, 1998) and the Union Carbide 


disaster in Bhopal (Fortun, 2001) providing two of the most well-known 


cases. As the critique starts from the idea that both the expertise and the 
interests — the two are seen as inseparable — of particular social groups have 
been ignored, the solution that is invariably proposed is the creation of more 
inclusive decision making forums in which a wider cross-section of 


communities can be represented (e.g. Douglas, 2009; Schot & Rip, 1997). 


Sociological interest in the enactment of expertise thus fits neatly with 


arguments for the democratization of expertise (e.g. Funtowicz & Ravetz, 
1993; Jasanoff, 2003; Maasen & Weingart, 2008). 

A different way of understanding this critique of technocracy is through 
the idea of “epistemic injustice.” First coined by Miranda Fricker (2007), 
“epistemic injustice” refers to the wrong that is done when there is a refusal 
to recognize a “bona fide knower,” or community of knowers. Epistemic 
injustices can be inflicted on both high and low status groups and, although it 
is true that many marginal or low status social groups continue to have their 
knowledge unfairly discounted, the same is now happening to higher status 


professions and institutions (Collins, 2014; Collins & Evans, 2002; Prior, 


2003). Well-known examples of this phenomenon include controversies over 
vaccines, in which parents reject consensual medical evidence (Boyce, 2006; 
Laurent-Ledru, Thomson, & Monsonego, 2011; Sheldon, 2009) and the 
decision of the South African government in 1999 not to use AZT to reduce 


the risk of mother-to-child transmission of HIV (Nattrass, 2012; Weinel, 


2007). Focusing purely on the enactment of expertise makes it difficult to 
offer a critique of these events because, within the constructivist framework 
that characterizes this approach, the legitimate sources of expertise are 


whatever the relevant social actors take them to be. 


Expertise as Property 


The alternative is to see expertise as the property of an individual or group 
(Collins & Evans, 2007). By foregrounding the notion of socialization this 
approach links the acquisition of expertise to participation in the relevant 
social practices. Where there is participation — i.e. experience — there can be 
expertise; where there is no participation there can be no expertise because 
there has been no opportunity to acquire the relevant tacit knowledge. Known 
as Studies of Expertise and Experience (SEE), this approach allows the social 
scientist to examine the social groups in which individuals have participated 
and, on this basis, to conclude whether expert status has been wrongly denied 
or wrongly attributed and, in so doing, to challenge the conclusions reached 
by participants (Collins, 2008; Collins & Evans, 2002, 2014a). 


Expertise as Real 


Arguments about who is an expert and who is not are the stuff of professional 
life so how can an outside analyst claim to know that, say, “X really is an 
expert in A but Y is not”? Well-known approaches such as the “five stage 
model” of Dreyfus and Dreyfus provide a starting point by showing that it is 
possible to identify different levels of expertise within a domain of practice 
(Dreyfus, 2004; Dreyfus & Dreyfus, 1986). Under the Dreyfus approach the 


initiate starts as a novice, self-consciously applying explicit rules but unable 


to grasp the nuances of context, and gradually works toward the expert stage, 
in which he or she is able intuitively to identify the salient features of the 
situation and the actions that should follow. The expert is, then, an individual 
who has gradually acquired skills through sustained practice. The 
sociological twist to this phenomenological account is to specify social 
embedding in an expert community as a necessary condition for developing 
these skills. This leads to an enculturation or apprenticeship model of 
learning that can be theorized in several ways, including the “situated 
learning” or “communities of practice” associated with Lave and Wenger 
(Lave & Wenger, 1991), Thomas Kuhn’s idea of a scientific paradigm 
(Kuhn, 1962), and the idea of “form of life’ (Bloor, 1983; Winch, 1958; 
Wittgenstein, 1953) used in Science and Technology Studies and by SEE in 


particular. 

The common element in these models is that expertise is held in, and 
sustained by, the activities of a social group. Gaining the skills needed to 
become an expert means becoming a full and active member of that group 


and learning to act in ways that other group members will recognize as 


appropriate or, at least, not inappropriate, when actors are confronted with 
new or novel problems. In some cases, the expected standards are codified 
with acceptance as an expert predicated on “reproducible performances of 
representative tasks that capture the essence of the respective domain” 


(Ericsson, 2006, p. 3). Examples of this include professions like medicine or 


law, where entry is via formal examinations and any subsequent work is 
expected to maintain appropriate levels of competence. Likewise, athletes 
and artists display their expertise though consistently performing well in 
competitions and giving performances that go beyond what “normal” or 
“ordinary” people can achieve. 

In other cases, the requirements are much less explicit and what counts 
as the standard is shared tacitly and only revealed through the group’s 
practices. The paradigm case here is natural language speaking, where the 
rules of grammar are notoriously hard to specify, but all native speakers 
know how to create an intelligible utterance. In fact, this is closer to the 
general case as even formal rules need informal meta-rules to determine how 
they should be applied in each new context (Collins, 1990). Thus, “thou shalt 
not kill” is generally a good rule but there are exceptions (e.g. on a 
battlefield, in self-defense, in order to save another life, and so on) and these 
exceptions will need meta-meta-rules as well (e.g. is this self-defense?). 

This idea of rules as essentially incomplete is crucial to the sociological 
understanding of expertise. According to this view, the meaning of an idea or 
word is revealed in the way it is used (Bloor, 1983; Winch, 1958). This 


makes expertise both context-sensitive and dependent on tacit knowledge. 


The apprenticeship model of learning is then necessary because only 
socialization can enable the individual to share the collective understandings 


of the group and so develop the tacit skills needed to apply them in new 


settings (Collins, 2010; Collins & Kusch, 1998; Dreyfus, 1979; Polanyi, 
1962, 1966). 


The Periodic Table of Expertises 


If expertise is the outcome of successful socialization, then an individual’s 
expertises are the accumulation of what has been acquired from the social 
groups in which he or she is a successful participant (Collins & Evans, 2017). 
As different individuals participate in different combinations of social 
groups, there must be a distribution of expertises that reflects these different 
experiences (Evans, 2011). These ideas are captured in a typology known as 
the “periodic table of expertises” (Collins & Evans, 2007), which is shown in 
Table 2.1. 


Table 2.1 The periodic table of expertises. 


UBIQUITOUS EXPERTISES 


Interactive Ability 
DISPOSITIONS ‘ - 
Reflective Ability 


UBIQUITOUS SPECIALIST 
SPECIALIST TACIT KNOWLEDGE TACIT KNOWLEDGE 


EXPERTISES Beer-mat Popular Primary Source —_Interactional Contributory 


Knowledge Understanding Knowledge Expertise Expertise 


Polimorphic 
Mimeomorphic 


fee T_T 
EXTERNAL INTERNAL 
META- 


EXPERTISES Ubiquitous | Local — Technical Downward | Referred 
Discrimination Discrimination Connoisseurship Discrimination Expertise 


META- 


CRITERIA Credentials Experience Track-Record 


Source: Collins & Evans (2007). 


The structure of the table is given by different kinds of participation. 
Working from the top, the first two rows identify the society-wide ubiquitous 
expertises and dispositions (personal characteristics) that are the foundation 
of social life and enable socialization in specialist groups; the notion that 
there can be expertises that are the possession of every person in a society is a 
characteristic of SEE that marks it off from approaches that treat all 
expertises as specialist. For SEE, ubiquitous expertises are the precondition 
for the development of narrow specialist expertises and meta-expertises in 
what are commonly referred to as expert domains. The different types of 
specialist expertise correspond to common-sense understandings of expertise. 
The first three categories — beer-mat knowledge, popular understanding, and 
primary source knowledge — denote the kinds of understanding that can be 
achieved solely by using resources such as magazines, books, academic 
journals, Google, YouTube videos, and so on. As these sources do not 
involve direct interaction with specialist communities, none of the tacit 
knowledge that is unique to a specialism can be acquired. In contrast, the last 
two categories — interactional expertise and contributory expertise — depend 
on immersion in the relevant community as this is the only way to acquire the 
specialist tacit knowledge needed to become a more complete expert in an 
esoteric domain. 

Because high-level expertise is hard to acquire — for example, it is 


suggested that it takes thousands of hours of deliberate practice (Ericsson, 


Krampe, & Tesch-Rémer, 1993; Ericsson & Pool, 2016) — making judgments 
about experts in the absence of specialist expertise is an inevitable feature of 
moder society (Beck, 1992; Giddens, 1990). The meta-expertises row lists 
some of the ways in which this can be done. These include external social 


judgments based on more or less ubiquitous expertises that make no reference 


to the content of what is being judged and more informed internal judgments 
that require some familiarity with the domain in question. The final row of 
the table identifies criteria that might be used for identifying experts, with 


relevant experience being the best of a weak set. 


Using the Periodic Table of Expertises 


The typology provides a new language for describing kinds of expertise. For 
example, the traditional idea of an expert is that of a skilled practitioner who 
has mastered both the language and practice of a domain: he or she can do the 
practice, describe the practice, teach the practice, develop new ways of doing 
the practice, and so on. Using Table 2.1, this corresponds to a contributory 
expert and depends on extensive immersion in the domain of practice and 
long experience of consistently acting in ways that match the group’s norms 
and expectations. Reading a lot about the domain, even exposure to the most 
sophisticated descriptions or multimedia demonstrations is not enough to lead 
to the acquisition of contributory expertise. 

As explained, under the SEE approach expertise is not necessarily 
scarce. We must all be ubiquitous experts or there would be no societies. 
Even where the distribution of an expertise is not quite uniform — e.g. car 
driving — there may be so many contributory experts that the expertise does 
not seem like an expertise at all (at least, not until you sit in a car with a 
learner driver!). In contrast, other expertises — e.g. astrophysics — will be 
located within small groups, making them seem much more esoteric and 
“expert-like.” As the difference between the two is determined by the nature 
of the group and not the nature of the expertise, it is, crucially, a sociological 
not an epistemological distinction. 

Emphasizing socialization and the importance of tacit knowledge has 
implications for interdisciplinarity and team working more generally (Collins, 
2011). Because expertise is intimately related to practice, collaborations 


between social groups may require each to learn new ways of seeing and 


doing in order to understand what the other is seeking to achieve and why 
(Galison, 1997; Gorman, 2002). Moreover, because what needs to be 
explained to the other group is usually unpredictable, time and space will 
need to be created in order for each group to gain some expertise in the 
language and practices of the other (Fisher et al., 2015). 

Finally, where decisions require particular kinds of expertise it is 
necessary to ask who needs to be included and who should be excluded 
and/or encouraged to participate in other ways (Callon, Lascoumes, & 
Barthe, 2011; Collins, Weinel, & Evans, 2010; Douglas, 2009). Where 
contributions depend on high-level expertise, then contributory experts are 
the preferred participants; where contributions depend on more ubiquitous 
expertises, then restrictions on participation are far fewer (Collins et al., 
2010). 


Interactional Expertise and Embodiment 


The term “interactional expertise” first appeared in 2002, when it was used to 
describe the expertise sociological fieldworkers develop following prolonged 
immersion in the communities they study (Collins & Evans, 2002). The claim 
was that even if social researchers do not participate in the full range of group 
activities (e.g. criminologists do not commit crimes) they can still claim to 
understand those practices because they can talk about them in ways their 
respondents recognize as valid. This is not to say that the researcher’s own 
disciplinary understanding is irrelevant. The point is that social scientists can, 
if they choose, justify their research on the basis that they understand the 
actors’ perspective without practicing their practices. 

In fact, the intellectual roots of interactional expertise are found some 
years earlier in debates about the nature of artificial intelligence (AI). 
Phenomenologists, most famously Hubert Dreyfus, argued that AI will fail 
because computers do not have human-like bodies (Dreyfus, 1979). In 
contrast, Collins argued that AI would fail because computers could not be 
socialized into the relevant community (Collins, 1990). According to Collins, 
embodiment could not be the crucial factor because there were many 
examples of disabled humans who were able to talk fluently about — i.e. 
display understanding of — all kinds of things that their bodies would prevent 
them from experiencing directly (Collins, 1996). 

Although the term “interactional expertise” is being applied to these 
debates retrospectively, it is clear that its defining feature — that it is possible 


to acquire an understanding of practice through deep immersion in the 


corresponding spoken discourse without actually engaging in the practice — 
maps onto the idea that it is the linguistic abilities of humans, not their 
bodies, that differentiate them from computers (for a more detailed discussion 
see e.g. Collins, 2000, 2004b; Selinger, Dreyfus, & Collins, 2007; Selinger & 
Mix, 2004). This philosophical debate about embodiment and language 


continues to this day in much the same terms (see e.g. Collins, 2016b). 

If the idea of interactional expertise is accepted, a new terrain of 
expertise-related research opens up. For example, how many interactional 
experts are there? The initial suggestion was that there would be very few, 
principally social researchers or specialist journalists whose work enabled 
them to sustain long-term interactions with a particular social group and 
hence become “special interactional experts.” This now seems like a serious 
underestimate. For example, within interdisciplinary collaborations that have 
a complex division of labor based on shared understanding, almost all of 
what each member understands about the others’ practices will come from 
interactional, not contributory, expertise (Collins, 2011). For example, the 
international team of gravitational wave physicists studied by Collins 
(Collins, 2004a, 2013a, 2017) were all specialists in particular subfields of 
physics but, in order to work together, they had to be able to understand what 
their colleagues in other subfields were doing. The novel insight that comes 
from SEE is that the scientists did not need to be practitioners (i.e. 
contributory experts) in those other subfields in order to understand the 
practices of those fields well enough to be able to work together. What they 
needed was the ability to understand the discourse of those subfields. Note 
that acquiring interactional expertise is not the same as reading about a 
domain of practice in texts or on the Internet, it involves long and deep 


immersion in the flux of linguistic discourse; such immersion is a means of 


acquiring tacit knowledge. This, incidentally, resolves the enigma of how it is 
possible for management of technical domains to work; to understand the 
domain they are managing, and to make recognizably good technical 
judgments within that domain, managers do not have to be front-line practical 
experts (Collins & Sanders, 2007). 

Outside workplace settings, the same problem arises: how do members 
of one social group come to understand the experiences of other groups? In 
some cases this will be via representations in the media or other forms of 
explicit knowledge but in others, and particularly where genuine 
understanding is needed, it will be best achieved by talking deeply to 
members of those groups about their experiences (Collins & Evans, 2014b). 
That said, the difficult and time-consuming process of acquiring a high level 
of interactional expertise in an esoteric domain should not be underestimated 
(Collins & Evans, 2015). For attempts to widen the notion of interactional 
expertise so as to make it less difficult to acquire see Goddiksen (2014) and 
Plaisance and Kennedy (2014). For resistance to such moves see Reyes- 
Galindo and Duarte (2015) and Collins, Evans, and Weinel (2015). 

These ideas have also given rise to a new research method based on the 
Turing Test (Collins et al., 2017; Collins, Evans, Ribeiro, & Hall, 2006; 


Evans & Collins, 2010). Known as the Imitation Game, the method explores 


both the content and distribution of interactional expertise by asking 
participants to pretend to be members of a different social group referred to 
as the “target group” (e.g. men might pretend to be women). Members of the 
target group (women in this example) then compare these answers to answers 
provided by genuine members of the target group to the same questions and 
try to work out which were produced by the pretender and which by the real 


women. Examining the questions asked and how they are answered by each 


group reveals what members of a social group think defines their culture and 
how well this is understood by other social groups. Research to date has 
included physiological topics such as color-blindness (Collins et al., 2006) 
and sociological topics such as gender and sexuality (Collins & Evans, 
2014b). It has also been used to explore the extent to which medical 
practitioners are able to take the patient’s perspective (Evans & Crocker, 
2013; Wehrens, 2014) and how well social scientists understand their 
fieldwork setting (Collins, 2016a; Giles, 2006). 


Three Dimensions of Expertise 


The sociological model of expertise as social fluency put forward in this 
chapter can be summarized as operationalizing expertise along three different 
dimensions (Collins 2013b): 


1. Individual accomplishment: this is similar to the approach of the stage 
models championed by such as Dreyfus and Dreyfus (1986) or Chi 
(2006) and captures the proficiency of the individual or group with 


respect to the domain of expertise in question. 


2. Esotericity: this captures the extent to which access to the social 


collectivity that holds the expertise is open or closed. 


3. Exposure to tacit knowledge: this describes the type of exposure that 
the learner has to the domain, ranging from published sources, through 


linguistic interaction to full participation. 


Combining these gives the three-dimensional expertise space shown in Figure 


2.1, in which domains of expertise and individual practitioners can be placed. 


1: 5-year-old 
language speaker 

2: Accomplished 
adult language 
speaker 

3: Novice sociologist 
Esoteric 


Ubiquitous 


4: Poet (virtuoso) 


Individual 
accomplishment 


Access to tacit knowledge of 
domain 


Figure 2.1 Three dimensions of expertise. 


This model allows a much richer description of the types of expertise 
available and the ways in which they develop and spread over time than the 
one-dimensional view of expertise as the mastery of esoteric skills. For 
example, it is possible to see how some expertises can be ubiquitous (e.g. 
natural language speaking), whilst others are esoteric (e.g. the virtuoso poet). 
It also enables individual accomplishment and practice to be set within a 
broader scheme that suggests what kind of practice is needed — thus, if full 
mastery means reaching the right-hand edge of the back wall, then practice 
must involve social interaction with the relevant community as, without this, 


the tacit knowledge of the domain can never be attained. 


Summary 


In this chapter we have focused on the analysis of expertise as a property of 
social groups that is acquired by individuals through their participation in 
those groups. The success of this process is measured by the increasing social 
fluency of the learner as they acquire the tacit knowledge needed to use their 
knowledge in ways that other group members recognize as correct. 

We have also argued that it is possible to distinguish between different 
types of expertise based on these different socialization experiences. Within 
this classification, the distinction between contributory and interactional 
expertises, and hence between the linguistic and physical aspects of 
socialization, reaches deep into philosophical debates about the embodiment 
of human knowledge. Although these debates continue, the model of 
expertise we defend suggests that socialization not embodiment is what 
underpins expertise. This, in turn, leads to a range of more practical and 
normative suggestions in which participation in the linguistic discourse, 
rather than the physical practice, of the relevant community is the key 
determinant of whether or not an individual knows what they are talking 
about. Practice, of course, remains the key to practical accomplishment but it 


is no longer the only source of good judgment. 
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Introduction 


Previous studies of expertise and expert performance have led to substantial 
gains in understanding, through fine-grained analysis of knowledge and skills 
usage among experts, including in comparison with novices. These studies 
have given rise to a model comprising cognitive structures in the mind, which 
organize knowledge and skills in performance. These cognitive structures 
have been seen to underlie expertise and its development. This “cognitive 
turn” in psychology and elsewhere (e.g. see Fuller, 1984; Fuller, de Mey, 
Shinn, & Woolgar, 1989; Hauptmann, 2011; Teubert, 2010) superseded the 


behaviorist approach that preceded it. The focus of behaviorism on 


investigating how a stimulus or set of stimuli causes a response in the form of 
behavior (SR) was replaced with studies that sought to identify cognitive 
structures that cause the behavior in question (CSR). Accompanying this 
move was a shift in focus from external stimuli to cognitive structures in the 
mind that represent the external world and mediate performance. 

Although this cognitive turn convincingly undermined the behaviorist 
notion of the mind as a “black box,” cognitive research to date has not 
convincingly addressed the question of how the gap can be bridged between 
contents of the mind and performance. In other words, how does mental 
content “leap out” into expert performance (Heidegger, 1927/1962, p. 87)? 

Due to strong early ties between psychology and education, a focus on 
knowledge and domain-specific skills carried over into schooling and 
educating for expertise in professional practice (Dall’ Alba 2009; Dall’ Alba & 


Barnacle, 2007). This focus on acquiring knowledge and skills organized in 


cognitive structure has not fulfilled the many expectations that were held of 
it, however. For instance, knowledgeable and skilled experts have been 
unable to simply transfer this expertise to keen novices, although this is not to 
deny the central role that effective teaching can play in promoting learning. 
Natural scientists, too, have more recently recognized that a focus on 
knowledge and skills in the teaching of science does not necessarily lead to 
good science. As a consequence, there have been moves to educate aspiring 
scientists beyond acquisition of knowledge and skills toward “thinking like 
scientists” (e.g. see Carillo, Lee, & Rickey, 2005; Hake, 2012). An additional 
conundrum for the cognitive turn in expertise research has been that the 
understanding displayed by some novices in performing complex tasks is 
more similar to experts than to other novices (Carter, Cushing, Sabers, Stein, 
& Berliner, 1988; Dall’ Alba & Sandberg, 2006; Livingston & Borko, 1989). 


How can we advance understanding of expertise and its development to 


account for these challenges? Can the challenges be adequately addressed 
within current theory? In recent decades, some expertise research has been 
grappling with the observation that expertise and expert performance appear 
to incorporate not only high levels of knowledge and skills, but also 
something else besides. What is this something else and how can it be 
developed? Or rather than something else in addition to knowledge and skills 
organized in cognitive structure, could expertise principally consist of 
something other than knowledge and skills in cognitive structure? 

Given the challenges to current theory, my aim in this chapter is to point 
to a way forward in reframing expertise and its development. This reframing 
necessitates adopting assumptions that differ from those prevalent within a 
model of cognitive structures in the mind or, in other words, it assumes a 


different ontological ground. Below I outline a theoretical framework that 


opens new avenues for researching expertise and its development, while also 
contending with the challenges noted above. This framework has as its source 
the notion of the lifeworld, or entwinement of life with world, from 


phenomenology, as I outline in what follows. 


Rethinking Expert Knowledge and Expertise 


In much conventional expertise research, experts are seen aS possessing 
and/or applying extensive knowledge in their domain of expertise. Experts, 
their knowledge, and domains of expertise are typically seen as separable and 
independent of each other. This assumption of separateness can be called into 
question, however. It makes no sense to talk of expert knowledge in the 
absence of experts or domains of expertise. For instance, expert knowledge 
about teaching only makes sense in the context of learners who are being 
taught by teachers. In contrast to an assumption of separateness, we can think 
of them as comprising an inescapable entwinement (Dall’ Alba, 2009). 

The concept of the lifeworld, proposed by Edmund Husserl (1936/1970) 
who is considered the founder of phenomenology, captures this inevitable 
entwinement of persons with their world. Rather than either external stimuli 
or cognitive structures in the mind, the lifeworld highlights the inescapable 
relation between persons and world. In other words, we are always already 
entwined with others and things in our world; we cannot step outside this 
entwinement with world. The lifeworld is not limited, then, to either an 
“inner world” of cognitions or an external world of stimuli. Instead, it is 
simultaneously my world and a world shared with others and things—a world 
we individually and collectively inhabit. The lifeworld is the everyday world 
we take for granted and from where all our endeavors arise, so it is both pre- 
scientific and pre-reflective. This concept of the lifeworld is common to the 
various branches of phenomenology that developed following Husserl, each 


with its own knowledge interest (see Spiegelberg, 1982). 


The concept of the lifeworld is further developed through Martin 
Heidegger’s (1927/1962) notion of “being-in-the-world.” Heidegger 
considered our domains of knowledge, such as architecture, history, and 
visual arts, as ways of being-in-the-world (p. 408). In other words, we enact 
our knowing in our ways of teaching, engineering, nursing, and so on. This 
emphasis directs attention not simply to an individual expert’s knowledge 
and skills, but to what is entailed in being expert in a specific domain. Being 
expert is not limited to knowledge and skills of individuals, but expert 
performance is inseparable from individual and collective activities, 
concerns, equipment, and endeavors in particular domains. So, the relation 
between experts and their domain of expertise is again highlighted, albeit 
with an ontological emphasis on being expert in a particular domain, as 
further elaborated below. 

While Heidegger gave the concept of the lifeworld an ontological 
emphasis, Maurice Merleau-Ponty (1945/1962) proposed that our being in 
the world is made possible through the perceiving body. Highlighting the 
relation between person and world, he argued that “consciousness is being- 
towards-the-thing through the intermediary of the body ... and to move one’s 
body is to aim at things through it; it is to allow oneself to respond to their 
call, which is made upon it independently of any representation” (pp. 
138-139). He pointed out that “movement is not thought about movement, 
and bodily space is not space thought of or represented” (p. 137). 
Emphasizing the mediating role of the body, he argued that “the body is the 
vehicle of being in the world” (p. 146). This body is not limited to 
interrelated systems of organs, but is the body as lived, as continually 
engaged in the world. 


For Merleau-Ponty, “perception and action are therefore essential 


collaborators with each other from our first embodied moments” which 
“dissolves the traditional conceptual split between the mental and the 


material” (Scully, 2012, p. 145). Merleau-Ponty’s concept of the “lived 


body” overcomes the problem of a gap between contents of the mind and 
expert performance. It shifts the focus of attention from the construct of 
“mind” to bodily perceptions, sensations, and movement about the world, 
which form the basis for habituated actions and, subsequently, for thought. 
For instance, “for us to be able to conceive space, it is in the first place 
necessary that we should have been thrust into it with our body” (Merleau- 
Ponty, 1945/1962, p. 142). 

Over time, bodily perceptions, sensations, and movement develop into 
habits based on experience of the material, socio-cultural world. These habits, 
enacted through the body, allow us to make our way about the world without 
being compelled to continually think in advance about everything we are 
doing. For Merleau-Ponty, “bodily actions or habits make thinking possible 
in the first place. And so the body and its habitual actions constitute forms of 
knowledge in themselves about how to be particular kinds of human beings 


in particular social settings” (Scully, 2012, p. 144). 


Drawing on Heidegger’s and Merleau-Ponty’s phenomenology, I argue 
that expertise and its development are not primarily dependent upon 
knowledge and skills in cognitive structure, but on embodied being in the 
world, inescapably entwined with others and things. In the section that 
follows, I explore what such a notion of expertise means for developing 


expert performance. 


Developing Expert Performance 


From a lifeworld perspective, expertise and expert performance are enacted 
and embodied, rather than possessed or applied as independent entities. 
Developing expertise relies upon this embodied being in the world. More 
specifically, developing expert performance requires integration of 
knowledge and skills into particular ways of being in the world (Dall’ Alba, 
2009), such as being economists, biologists, or social workers. This means it 
is insufficient simply to possess knowledge and skills, or to apply them to 
specific tasks or problems, however complex these may be. Possession or 
application of knowledge and skills is not sufficient to constitute expertise. 
Instead, knowledge and skills must become integrated into being expert, as I 
discuss below. 

If we seek to promote the development of expertise, then, what would 
this entail? An important consideration is enhancing not only what aspiring 
experts know and can do (an epistemological dimension), but also how they 
are learning to be (an ontological dimension) in relation to the domain in 


question (see Dall’ Alba, 2009, for elaboration). To date, research on expertise 


and associated efforts to develop expert performance have had largely an 
epistemological emphasis. Developing expertise requires, then, an 
“ontological turn” through adopting a relational focus (Heidegger, 1998; see 


also Barnett, 2004; Dall’Alba & Barnacle, 2007). Developing expert 


performance involves not simply increasing knowledge and skills, but 
integrating these knowledge and skills into expert ways of being in a specific 


domain. The inevitable entwinement of (aspiring) experts with their world is 


foregrounded. Developing expertise can be conceptualized, then, as a 
continuing process of becoming; never entirely complete, nor achieved once 
and for all. 

Simply disseminating knowledge and skills, even by enthusiastic 
experts, falls short of promoting such a process of becoming. Indeed, the 
problem of knowledge “transfer” has itself arguably been created through an 
assumption of separate, independent components of expertise. While efforts 
to educate aspiring scientists in “thinking like scientists” recognize 
limitations in disseminating knowledge and skills, they typically fall short of 
a clear focus on developing embodied ways of being scientists. 

In learning to enact what we know, we embody ways of being in the 
world. Knowledge and skills are formed and organized into embodied ways 
of being, which serve to direct further development of expertise. For 
example, when teaching is enacted as knowledge dissemination, efforts to 
develop expertise in teaching tend to concentrate on teachers’ presentation of 
content. Developing knowledge and skills that improve content presentation 
comes into focus. When teaching, instead, actively seeks to facilitate 
learning, developing expertise becomes concerned with monitoring and 
enhancing learning as it occurs (see, for example, Borko, Davinroy, Bliem, & 
Cumbo, 2000). The knowledge and skills that become integrated into 
facilitating learning center on understanding what learning involves for 
learners in a particular setting, how this learning can be monitored, and 
embodying what can be done to enhance this learning. Ways of being 
teachers such as these are enacted in how teaching is performed and 
developed, both individually and collectively. 

Current ways of being can present obstacles to achieving expert 


performance, even when extensive knowledge and skills are accumulated in a 


conventional sense (e.g. see Benner, Tanner, & Chesla, 1996; Borko et al., 


2000; Dall’ Alba, 2009). For instance, if expertise in teaching requires careful 
attention to learning, then primarily improving presentation of content stands 
in the way of enacting expert teaching, regardless of the extent of acquired 
knowledge or skill (e.g. see Borko et al., 2000). It follows, therefore, that if 
targeted, deliberate practice does not take into account current ways of being, 
it may be ineffective. Development can consist of merely reinforcing and 
refining existing ways of teaching, without the transformation that would be 
required in being teachers focused on monitoring and enhancing learning (for 
examples, see Borko et al., 2000; for examples in learning to be medical 
practitioners, see Dall’ Alba, 2009, and Dall’ Alba & Sandberg, 2006). 


Such qualitative differences in expertise — of the kind outlined above for 


teaching — form the basis for Hubert and Stuart Dreyfus’s (1986) influential 
stage model of development from novice to expert. However, this stage 
model does not account for differences that are evident at a single stage of 


development (see Dall’ Alba & Sandberg, 2006, for elaboration). Depending 


upon embodied experiences built up over time and responses to these 
experiences, novices can show similarities to expert ways of being, albeit 
usually less fluent and refined in the enactment due to fewer, less varied 
experiences. For instance, both expert and novice teachers can be attuned to 
learning by learners, although they are likely to differ in their repertoire for 
responding to those attunements. 

When we identify experts in a particular domain, we expect them 
consistently to demonstrate more accomplished performance than most 
others. Indeed, such high-level performance is inherent in the notion of being 
expert (see also Ericsson, 2006; Sonnentag, Niessen, & Volmer, 2006). We 


also recognize that some never attain this expert status, despite extensive 


knowledge or years of experience (Benner et al., 1996; Dreyfus & Dreyfus, 


1986). Being expert entails, then, something other than simply increasing 
knowledge and skills. It requires consistently demonstrating high-level 
performance through responding in attuned ways to the particular setting and 
issues at hand. This “attuned responsiveness” (Dall’Alba, 2009, p. 68) 
includes taking an informed and responsible stand on the varied situations 
and issues encountered. It would be a contradiction in terms to speak of an 
uninformed or irresponsible expert. We expect experts to display not only 
substantial knowledge and skills, but also a capacity for critical reflection, 
while exercising responsible judgment. In other words, we expect not only 
expert knowledge, but expert ways of being. 

Such attuned responsiveness to the circumstances and issues at hand is 
not readily addressed through standardized performance measures, as evident 
in the risk of teaching to the test in ways that can limit learning. What is 
considered to be attuned, high-level performance in particular settings also 
alters over time, aS expertise and its related domains develop. For instance, in 
recent years, promoting literacy in the use of information and communication 
technologies among learners has increasingly become an expectation held of 
teachers in schools. As developments occur in domains of expertise over 
time, it follows that standardized, generic measures of performance cannot 
fully capture performance once and for all, as controversy over efforts to 
measure intelligence has also made manifest (e.g. see Kaufman, 2009). 

Ways of being in a particular domain also show some variation from one 
socio-cultural, material context to another. For instance, the teaching required 
of a teacher with chalk, a blackboard, and a few books in a remote village 
contrasts with the teaching demanded in a richly resourced virtual 


environment. Similarly, ways of being teachers can vary from one classroom 


to an adjacent classroom. This is because expertise is dynamic, embodied, 
intersubjective, and plural, in line with the inseparable relation between 
persons and world. A limitation of expertise research carried out in 
controlled, laboratory conditions with standardized tasks is that it does not 
account for this plurality (as Micheline Chi acknowledges: Chi, 2006; see 
also Clancey, 2006; Ross, Shafer, & Klein, 2006; and Norman, Eva, Brooks, 


& Hamstra, 2006, on the importance of the situation or context). In the 


everyday world, variations occur as situations alter and requirements change. 
At the same time, if it is to be considered teaching, there will be some 
commonalities across time and settings, such as teachers, learners, and 
material to be learned with relevance to the context. 

It is crucial, then, that developing expertise attends not only to 
knowledge and skills, but also to attuned responsiveness to others and things 
in the surrounding environment, across variations in time and place. In other 
words, ways of being in the world are to be in focus through integrating 
knowing, acting, and being experts in the domain in question (Dall’ Alba, 
2009; Dall’ Alba & Barnacle, 2007). 


Concluding Remarks 


In some respects, both behaviorism and the later cognitive turn each pointed 
to important features of expertise. On the one hand, behaviorism indicated the 
perceived world (while underestimating the perceiving body). On the other 
hand, the cognitive turn signaled active involvement of the mind (while 
downplaying the world calling forth this response). These are two sides of a 
common coin, each incomplete in its own way. 

A lifeworld perspective from phenomenology calls into question the 
usefulness of cognitive structures in the mind as “a theoretical construct, a 
model of a possible interface between body and behaviour” (Teubert, 2010, 
p. 43) for contending with the challenges to theory in exploring expertise into 
the future. This model displaces attention from the embodied way in which 
expertise is enacted in addressing specific tasks and concerns in particular 
settings, while creating an unbridgeable gap between mental content and 
expert performance. This gap is removed when a lifeworld perspective is 
adopted. Similarly, a relational, ontological perspective on ways of being in 
the world exposes the problem of knowledge “transfer” as an inadequate 
conceptualization, while demonstrating multiplicity in ways of being at any 
given level of experience. 

In order to advance understanding of expertise and its development, the 
earlier shift in focus from external stimuli to cognitive structures in the mind 
arguably now demands a shift in ontological ground, toward recognizing the 
centrality of the inescapable relation between persons and world. The notion 


of the lifeworld in phenomenology highlights this inevitable entwinement of 


persons with world, providing novel resources for further research and 


development in expert performance. 
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Introduction 


Darwin’s (1859) theory of evolution by natural and sexual selection has the 
potential to organize and unify the social sciences, including our 
understanding of human expertise. From an evolutionary perspective, humans 
have long developed “expertise” in functional domains such as hunting or 
toolmaking; however, more recently in our species’ history, humans began to 
develop expertise in arguably non-functional domains such as chess, sports, 
or art. The development of expertise in functional domains appears easier to 
explain than the development of expertise in domains that are further 
removed from survival and reproductive success. Even still, expertise as 
defined in modern psychology was likely not necessary for survival and 
reproduction. Our ancestors needed to be good enough hunters to survive but 
did not need the same degree of expertise that is often developed in modern 
contexts (e.g. violinist, chess master) where many thousands of hours are 
devoted to single, evolutionarily obscure tasks. Modern researchers define 
expertise as the mastery of specific skills that allow for performance that is 
beyond one’s peers (Ericsson, 2014). To be an expert, in the modern sense, 
an individual must consistently produce results (e.g. winning in chess 
matches) that can be, in principle, measured and replicated in the lab 
(Ericsson, Prietula, & Cokely, 2007). Thus, real expertise is not simply 
possessing skill or fluency; it involves extreme skill that allows for consistent 
performance over other potentially skilled conspecifics. This leaves us with 
the questions of why develop expertise at all and why in seemingly arbitrary 


modern domains? We propose that modern-day expertise emerges from a 


confluence of factors, including the plasticity of evolved cognitive abilities, 
increasing social competition, and an inherent motivation to signal desirable 
traits in areas that are culturally valued and tied to social prestige. 

We open with discussion of Alexander’s (1990) ecological dominance 
and social competition model of human evolution, and how the ecological 
dominance of humans contributed to cognitive and behavioral plasticity and 
set the stage for the development of evolutionarily novel domains, such as 
chess (Geary, 2005, 2007). The combination of novel domains, social 
competition, and social signaling creates an arms race that drives the 
development of expertise in seemingly non-functional domains, as discussed 


later in the chapter. 


Ecological Dominance and Social 
Competition 


Although we may not be the only animals to develop and propagate culture 


(Lycett, Collard, & McGrew, 2009), our culture is certainly more 


complicated and sophisticated than that of any other animal. Humans are also 
uniquely cooperative primates. Unlike other primates, humans form broad 


multilevel networks with kin and non-kin (Chapais, 2010). They transmit 


information in unusually complex ways (e.g. language) and do so, in part, to 


coordinate group activities and divide into specialized roles (Geary, 2005; 


Pinker, 2010). These peculiar capacities likely allowed a small and 


unintimidating ape to spread into multiple continents and to cope with 
disparate ecologies, from the unforgiving winters of the north to the sun- 
beaten deserts and humid forests of the equator. Scholars have put forward 
many theories to explain the evolution of humans’ novel nexus of traits. We 
believe that Alexander’s (1990) ecological dominance and social competition 
theory (EDSC) is among the best (see also Flinn, Geary, & Ward, 2005; 
Geary, 2005). 


According to EDSC, at some point in hominid evolution, our ancestors 


achieved a significant ability to control ecological selection pressures such as 
pathogens, climate, and prey. They did this in myriad ways. For example, the 
control of fire better allowed humans to avoid consuming pathogens, gave 
them rapid access to nutrient-rich foods, and may have served as a deterrent 


to predators, especially at night (Wrangham & Carmody, 2010). Humans also 


learned to make more effective tools which increased hunting and fishing 


efficiency and also allowed for the making of better shelters (Ambrose, 
2001). The reduction of these ecological pressures allowed for rapid 
population growth and changed the central target of selection forces from 
Starvation, disease, and predation to social pressures, such as ability to learn 
rapidly and outsmart conspecifics (Geary, 2005). That is, at some point our 
ancestors became their own chief selective forces, competing intensively 
against each other (sometimes by cooperating in groups) for resource control 
and social influence. The resulting within-species arms race is arguably the 
best explanation for the unprecedented increase in hominid brain size (Bailey 


& Geary, 2009), and the human cognitive and social competencies that 


differentiate us from other species. 
Researchers have proposed that one key result was the expansion of 
social, behavioral, and cognitive plasticity to cope with variation and change 


(Geary, 2005). Other people’s unpredictable and self-interested behaviors are 


the most critical sources of this variation. To anticipate and cope with fluid 
social dynamics, humans have evolved the capacity to generate mental 
models or run imaginary scenarios: “If he does this, and I do that, then this 
will happen.” Moving up a social hierarchy is very different from extracting 
fruit from a tree. The second problem is stable across generations and within 
lifetimes and thus evolution could solve it with a system of rigid, 
straightforward algorithms (hard modules). The first problem, though, 
changes with different constellations of people and groups of people, and 
individuals with rigid, straightforward social algorithms would be quickly 
outmaneuvered. Geary (2005, 2007) outlined how this arms race and the 
attendant use of mental models, combined with modular plasticity (below), 


could have resulted in the evolution of general fluid intelligence and the 


ability to create and learn evolutionarily novel things, such as writing systems 
and chess. 

An alternative, but not necessarily mutually opposed perspective, posits 
that environmental variability was an important driver of human cognitive 


capacity and behavioral flexibility (Potts, 1998). Variable environments place 


strains on organisms and select for behavioral plasticity and can lead to 
adaptive versatility (Potts, 2013). Some evidence supports the hypothesis that 
climatic variation contributed to human brain expansion (Shultz & Maslin, 
2013). In this view, the attainment of high quality food did in fact place 
tremendous cognitive stress on hominins because the environment changed 
relatively rapidly, thus a hard modular system would lead to inefficient 
foraging and place less intelligent and behaviorally inflexible individuals at 
greater risk for starvation. 

Whatever the exact truth of these perspectives, and we suspect both 
ecological climate and social climate played a role in human uniqueness (see 


Bailey & Geary, 2009), the end product is the same: an organism that is 


uniquely intelligent, able to learn novel skills, and to behave flexibly. An 


organism, in other words, that has entered the cognitive niche (Pinker, 2010). 


Models of Human Cognition 


For much of the twentieth century, many thinkers assumed human minds 


were extremely malleable (Pinker, 2002). This “blank slate” perspective was 


popular among behavioral psychologists and in the social sciences more 
generally. In the 1950s, the blank slate view was questioned by a number of 
psychologists, philosophers, and linguists, ushering in the cognitive 
revolution. Notable in the ensuing paradigm shift was Fodor’s (1983) 
argument that the mind was probably composed of computationally distinct 
mechanisms, each devoted to solving specific problems in straightforward, 
algorithmic ways. This gave rise to a “hard modularity” approach to 
cognition, whereby the mind is composed of a system of unique modules, 
each using specific algorithms to solve evolutionarily recurrent problems, 
such as detecting predators and social cheaters (e.g. see Tooby & Cosmides, 
i592). 

We believe that neither the blank slate nor the hard modularity 
approaches are sufficient for understanding expert performance. Rather, a 
“soft” modularity approach, one that relies upon a distinction between 
primary (evolved) and secondary (learned, culturally specific) competencies 
affords a more nuanced understanding of the human mind (Geary, 1995; 
Geary & Huffman, 2002). On this view, the human mind is composed of 
basic modular functions that support universal, primary competencies. These 
primary systems require fleshing out and adaptation to local conditions 
through children’s engagement in play, exploration, and social discourse, 
forms of naturally occurring practice. The eventual results are systems of 


cognitive competencies organized around the domains of folk psychology, 


folk biology, and folk physics, as shown in Figure 4.1. These systems enable 
people to cope with universal social (e.g. developing relationships) and 
ecological demands (e.g. hunting and navigation), but can be modified 
(within constraints) through developmental experiences to accommodate the 


many different social and ecological niches in which humans are situated. 


Figure 4.1 Evolutionarily salient information processing domains, and 
associated cognitive modules that compose the domains of folk 


psychology, folk biology, and folk physics. 


Adapted from D. G. Geary, The origin of mind: Evolution of brain, 
cognition, and general intelligence, 2005, p. 129. © 2005 American 


Psychological Association. 


The plasticity of these systems follows from ecological dominance, 
climatic variability, and the resulting expansion into novel ecologies and 
increasing competition from sophisticated conspecifics. Although much 
remains to be determined, this plasticity likely involves the sensitivity of 
these systems to a wider range of information than is found in other species 


and the ability to link systems in top-down and in novel ways (Geary, 2005). 


The top-down modification of soft modules is supported by the attentional 
control components of working memory, abstract problem-solving that is a 
key feature of fluid intelligence, and through practice. The result is the 
potential to develop evolutionarily novel, secondary competencies. For 
example, the ability to write poetic verse in iambic pentameter with end 
rhymes is a secondary competency that is possible through explicit, top-down 
modification of primary language abilities. These natural rhythms of speech 
provide the scaffolding for many forms of poetry, the writing of which 
requires explicit control over language production and multiple cycles of 
revision to achieve the desired effect. Perfecting the ability to write this form 


of verse likely requires considerable practice. 


Secondary Competencies and Expertise 


Expert performance as studied by psychologists is largely focused on 


secondary domains (e.g. chess, music) (Ericsson & Charness, 1994), and 


even in domains more closely related to primary abilities, the level of skill 
development is unusual from an evolutionary perspective (Epstein, 2014). 
Expert performance requires deliberate practice to achieve; it does not appear 
to result from the natural play and exploratory behaviors that flesh out 
primary systems. However, as noted, expert performance does build from the 
scaffolding of basic modules and primary competencies. Chess, for instance, 
requires the ability to think about space, movement, time, and to hold a goal 
in mind, and simulates group-level military conflict and strategy. Chess 
builds from primary competencies and motivations in folk physics and folk 
psychology (i.e. “what is my opponent going to do?”), but pulls these 
together in a novel way. The achievement of chess expertise requires some 


level of fluid intelligence and attentional focus (Campitelli & Gobet, 2011), 


along with significant, controlled practice with reliable feedback. By analogy, 
deliberate practice is to secondary competencies (and expertise, by extension) 
what play and exploration are for primary competencies (Ericsson, 2008). A 
key difference is that many of the attentional, cognitive, motivational, and 
behavioral biases that support the fleshing out of primary systems are built in, 
but these have to be largely constructed through deliberate, goal-directed 
activities for the development of competencies in secondary domains. 

It is probable that expert performance, like the weather, is dramatically 
affected by small variation in input variables. That is, small differences in 


initial physical or cognitive abilities or more likely constellations of them are 


probably exaggerated as performers reach elite levels (Epstein, 2014). 
Consider, for an obvious example, height and success in basketball. In lower 
levels of competitive play, individual differences in height, although 
important, are not determinative. There are many relatively short players in 
division one college basketball compared to the NBA (National Basketball 
Association), and individual differences in height probably do not matter as 
much. For example, a successful center in college basketball does not have to 
approach seven feet tall. Productive NBA centers in contrast are typically 
close to seven feet tall. Today, for example, the shortest center among the top 
ten in player efficiency, which assesses the impact a player has on the game, 


is six foot ten inches (ESPN Hollinger Ratings, 2015). What is true for height 


is probably true for less conspicuous traits such as spatial perception, 
hand-eye coordination, and short-term memory, among others. Even 
apparently irrelevant traits such as wrist tendon flexibility might greatly 
affect the capacity of a person to throw a slider or a curveball. And the 
difference between an easy to hit and a devastating curveball is probably not 


large (Epstein, 2014). As we document below, this has implications for 


thinking about the development of expertise in an evolutionary perspective. 


Expertise and Social Signaling 


As noted, the development of many forms of expertise appears without 
evolutionary function. It is hard to understand why a man might practice for 
thousands of hours just so he can hit a table tennis ball better than another 
man, allowing him to win 53 percent of his games. And this is especially 
difficult to understand because the man could be using that time to achieve 
other goals such as searching for a mate, socializing with friends, working at 
the office — all of which are more immediately relevant to the evolutionary 
imperatives of mating and survival (Buss, 1995). We believe that signaling 
theory can help explain this apparent mystery. In other words, we believe that 
expert performance is a signal that can attract social partners, romantic 
partners, and prestige because in modern contexts it communicates the 
possession of desirable traits, skills, or, even, genes. 

Signaling theory explains the logic of animal communication from 
Ornamentation to vocalizations and many previously enigmatic phenomena 
such as gaudy nests and time-consuming courtship dances (Searcy & 
Norwicki, 2010; Zahavi & Zahavi, 1997). The basic tenets are 
straightforward. Trait quality varies among individuals of all sexually 
reproducing species. The quality of such traits is not always easy to perceive, 
but trait quality can be reliably indicated by another more perceivable trait. 
Therefore, both perceivers and senders of the signal can benefit from it; high 
quality signalers benefit by indicating the quality of their traits, and receivers 
benefit by being able to discriminate between high and low quality 


individuals. However, signalers can potentially dissemble high quality traits 


by enhancing their signals without changing the quality of their underlying 
trait; therefore, receivers of the signal must remain vigilant against deception. 
A solution to cheating is the development of costly or hard-to-fake signals, 
those that impose costs that would-be deceivers cannot bear. And this 
explains why many signals, especially those sent among animals with 
potentially competing interests, are elaborate: such signals are costly and 
therefore make cheating difficult (Zahavi, 1975). 

In humans, costly signals are especially important because many 
culturally important competencies or the ability to develop them are non- 
physical and thus difficult to directly evaluate. We believe that expertise (or 
elite performance) often, but not always, functions as a costly signal of some 
desirable underlying trait (Miller, 2001; Winegard, Winegard, & Geary, 
2014). Consider these features of expert performance that make it a good 
candidate for a costly signal: (1) expert performance is often broadcast 
publicly; (2) there are enormous individual differences in the domains in 
which people care about expert performance (music, sports, art), making for 
obvious rankings between competitors or performers; (3) performances are 
generally ritualized or organized in such a way that they can be assessed, also 
facilitating ranking of performers; and (4) expertise is difficult to achieve and 
quite rare, meaning it is costly and that it relies upon unique constellations of 
underlying traits and large amounts of leisure time. These traits may consist 
of, but are not limited to, conscientiousness, athleticism, intelligence, the size 
of one’s social network, and ambition (Hawkes & Bliege Bird, 2002; 
McAndrew, 2002; Miller, 2001). The signal value of expertise and an ever- 
expanding population of potential competitors create another within-species 
arms race that plays out within rather than across lifespans. 


Notice a few things that follow from this signaling analysis. The first is 


that expert performance can only occur in domains in which individual 
difference variables (including deliberate practice) predict outcomes in 
performance. There is no such thing as expert performance in tic-tac-toe and 
other simple games, because high levels of performance are easy to achieve 
and thus are a weak signal of individual differences in any underlying traits. 
And second, the prestige that accrues to those who develop expertise 
motivates people to dedicate the many grueling hours necessary to develop it 
(Davis & Moore, 1945; Henrich & Gil-White, 2001). Of course, there are 


other motivating factors, but the prestige that experts obtain, which can 


improve resource control (e.g. pay) and social influence is certainly a large 


one, especially in domains valued by the culture (Geary, 2010). Indeed, there 


is evidence that early humans gained social capital and other resources 
through the possession of expertise in skills such as flintknapping, ceramic 


production, and handaxe production (Ferguson, 2008; Olausson, 2008). This 


appears to be due to a prestige bias in other human learners who then defer to 
experts and grant them status in exchange for the valuable knowledge that the 
experts possess (Henrich & Gil-White, 2001; Mesoudi, 2008). 

In general, we would expect people to pursue their comparative 
advantage when deciding which skills to develop; and we suspect that the 
pursuit of expertise is not much different from “normal” skill acquisition in 
secondary domains (Geary, 2007). Consider two people who show the same 
initial ability to shoot three pointers in basketball. One is five foot four inches 
and incredibly intelligent. The other is six foot five inches and of average 
intelligence. Other things equal, the second person is more likely to choose to 
develop his shooting skills, deliberately practicing and forgoing other 
activities. Of course, there are many other causes of practice, including 


parental pressure. But, generally, people should pursue their comparative 


advantage, meaning that those who practice for many hours in one domain 
are not a random sample, because only relatively skilled individuals would 
maintain effortful practice in a domain he or she did not excel in. Note that, 
from an evolutionary perspective, an individual does not have to be the 
absolute “best” at a particular skill to gain prestige and resources. All that is 
required is that he or she is better than local conspecifics. It is only in our 
modern, evolutionarily novel, environment that individuals compete globally. 
Thus, an individual’s comparative advantage will be context dependent. 

If our perspective is on the right track, a straightforward prediction is 
that humans began to exhibit expertise in differentiated domains such as 
toolmaking, art, ceramics, etc. during the slow process in which humans 
achieved ecological dominance and began living in more variegated societies 
marked by a division of labor and specialization (Harari, 2015). Some 
evidence suggests that this prediction is accurate. While humans have been 
making tools for roughly 2.6 million years, the complexity of tools increased 
with the expansion of brain size and it was only 250,000 years ago that 
humans began producing late Acheulean tools, which required secondary 
competencies and the ability to use extensive self-control to achieve long- 
term objectives (Stout, Toth, Schick, & Chaminade, 2008). Around 
60,000—30,000 years ago, humans achieved “cultural modernity” which 
corresponds to a proliferation of symbolically mediated artifacts such as 
personal ornaments, engraved objects, flasks, paintings, and carved figurines 
(Conard, 2010; Wadley, 2001; Zilhao, 2007). The ability to create most of 


these artifacts required an apprenticeship system and deliberate practice — 


they are very complex artifacts and by no means intuitive to create (Stout, 
2011). Once modern human cognition was in place, the limiting factor on the 


development of expertise was social complexity. Those societies that had a 


deeper division of labor also created more experts and eventually developed 
some form of educational system to enable the deliberate practice required to 


gain expertise (Trigger, 2007) 


Conclusion 


We believe the most useful theory of human cognition and cognitive and 
behavioral plasticity is soft modularity, which likely evolved after our 
ancestors achieved ecological dominance and in response to attendant 
increases in social complexity and competition as well as greater ecological 
variability associated with range expansion. We used this to argue that 
expertise, as defined by modern psychology, largely occurs in evolutionarily 
novel secondary domains that necessarily involve top-down learning and 
practice. The levels of expertise needed in these domains — and even domains 
that are strongly dependent on primary skills (e.g. basketball) — are likely 
unprecedented and increasing, because the pool of potential competitors is 
ever increasing and economic specialization and leisure time provide 
opportunities to develop expertise in evolutionarily novel ways. Though 
expressed in potentially novel ways (e.g. chess, violin), expertise in these 
domains probably functions as a signal of underlying traits that motivates and 
garners prestige for those who can display it. The signaled traits may vary to 
some extent (e.g. visuospatial-based as in basketball or language-based as in 
poetry), but most generally signal the ability and willingness to engage in 
effortful, deliberate practice. The latter is likely a rare and valuable trait in 


and of itself. 
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Introduction 


Expertise has been suggested as an indicator of the emergence of 
consciousness in humans (Coolidge & Wynn, 2005; Rossano, 2003; Stout, 
2011). Unfortunately, this perspective presupposes expertise is absent in non- 
human animals (from here on simply animals). This view is likely untenable 
depending on the definition of expertise actually proposed. If expertise is 
considered the outcome of deliberate practice and by the term deliberate 
conscious intent is meant, then we face a dilemma. People only attribute full 
consciousness to adult human language users without controversy. No one 
would deny the normal state of adult intact people is the capacity for 
consciousness. Any other attribution of consciousness to any entity besides 
adult human language users is controversial. Until such a time as a neural 
correlate of consciousness can be independently measured and validated in 
humans and other animals, by definition other animals may be excluded from 
having expertise which requires consciousness. The use of expertise would 
then be unhelpful for determining when consciousness emerged in the 
evolution of our or any species (it would be viciously circular), unless 
expertise could be discerned with indicators other than conscious intent itself. 
Other definitions of expertise not reliant on internal state attributions seem 
more encouraging of a broader perspective of expertise which sees expertise 
potentially as widespread phenomena amongst animals (Helton, 2005, 2007, 
2008). Indeed, expertise and its development may be why animals have 
complex nervous systems. The chapter will relay alternative objective 


definitions of expertise, demonstrate animals can satisfy these definitions, 


explain the benefits of a wider comparative perspective, and present some 


interesting questions raised by animal expertise. 


Objective Definitions of Expertise 


Non-internal state attribution definitions of expertise have been proposed. 
These include defining expertise as exceptional performance, a social 
construction, or the outcome of prolonged learning (Helton, 2008). These 
definitions are detailed in the following subsections and we provide evidence 
of where at least some animals appear to satisfy the proposed definition. If 
issues of consciousness and deliberateness are put aside, then an exploration 
of animals’ expertise and its development may assist in our understanding of 
expertise and prove practically important. Indeed, examining animals may 


help with defining expertise or at least refining proposed definitions. 


Expertise as Exceptional Performance 


Many cognitive scientists propose that expertise is demonstrated, perhaps 


defined, by objective exceptional performance (Ericsson & Charness, 1994; 


Ericsson, Charness, Hoffman, & Feltovich, 2006). If expertise is open to 
scientific investigation it must be replicable and objective. One path to an 
objective definition of expertise is to base the definition on objective 
performance and then use a relative metric of performance to determine who 
(or what) is an expert. For example, if we examine sprinters we would have 
their objective running times; we could then scale the sprinters on relative 
ability and then determine some rough cut-off of expertise. This could be the 
upper 5 percent or even upper 1 percent. A plausible objection to this 
definition is that any distribution of performance domains would have an 
upper percentile of performers; this is a statistical fact. We would probably 
not want to define into existence expert spark plugs — those spark plugs 
which perform in the upper percentile. Although objective criteria for 
expertise are critical, we would need to limit the domain to systems that 
presumably acquire their expertise via a process of learning and one that 
presumably is not learned in a limited time (see the third definition regarding 
prolonged learning). Regardless of these challenges, would other animals 
qualify based on a performance definition? Undoubtedly for any animal skill 
we could rank-order animals for their performance and select the upper 
percentile of performers. This could simply reflect innate differences, so we 
would have to refine the definition to refer to skills that are acquired over 
time. In this case we would need to look no further than dogs (Canis lupus 


familiaris). Through training dogs learn to excel at a wide variety of tasks, for 


example, accelerant detection, blind assistance, epilepsy detection, explosives 
detection, forensic tracking, guarding, hearing assistance, herding livestock, 
medical diagnosis, narcotics detection, detection of insect infestations and 
microbial growth, sprinting, sled-pulling, and fighting (Serpell, 1995). These 
dogs are sorted on objective performance metrics. Those highly skilled are 


experts. 


Expertise as a Social Construction 


Some researchers would advocate that expertise is a socially derived label or 


in other words a social construction (Sternberg & Ben-Zeev, 2001). Indeed 


Agnew, Ford, and Hayes (1994) have argued that the minimum criterion for 
expertise is simply having a large group of people label the individual as an 
expert. While this social voting criterion may not appeal to all researchers, for 
those who advocate this perspective would some animals qualify? Yes, the 
evidence would support this conclusion. Dogs again serve as insightful 
candidates. First, societal laws have for a long time recognized the distinct 
value of highly skilled or expert animals. Indeed the Welsh laws of Hywel 


Dda as early as 945 ce imposed different penalties for killing trained and 


untrained dogs (Menache, 2000). In the modern United States, most states 
impose much stiffer penalties for people who either injure or interfere with a 
service or law enforcement working dog than they would a pet (Randolph, 
1997). Indeed the United States Internal Revenue Service even recognizes 
dog expertise, as assistance dogs are a legitimate medical expense for tax 
deduction purposes (Treasury Regulation 1.213—1(e)(1)(iii)). Pets, however, 
are not deductible. The laws are clear: trained animals are different. Second, a 
variety of animals are given unique awards for their expert performance. 
Indeed, for military working animals, the British military even created a 
special award, the Dickin Medal, for exceptional performance and action. 
Theo, a British springer spaniel, who died in Afghanistan while serving in the 
British military, was awarded the medal in 2012 for exceptional performance 
and other exceptional animals (not just dogs) have been similarly recognized. 


Third, opinion surveys of other professionals indicate the high regard and 


recognition given to expert animals. Sanquist and colleagues (Sanquist, 
Mahy, Posse, & Morris, 2006), for example, surveyed 78 security 
professionals at Pacific Northwest Laboratories, a center for security 
research, and explosive detection dogs were rated the overall highest security 
measure available. 

Only language users (people) can vote or actually say an individual is an 
expert, hence, the above examples are of people providing indications that 
they socially label other animals as exceptional or expert. The examples 
provided are of dogs, but would hold true for other exceptional animals as 
well, such as some horses in equestrian sports. Amongst non-language using 
animals another possibility to consider is the choice animals may make when 
socially learning a skill. Laland (2004) argues that some animals learn from 
other animals using a “copy if better” strategy, suggesting some animals are 
capable of recognizing others more skilled than themselves. So another 
possibility is conspecifics, even if non-language users, could vote on who is 
an expert, because they are the ones they try to emulate. This could be 
explored further, but may be relatively rare amongst animals. Regardless, 


some non-human animals are socially constructed or labeled as experts. 


Expertise as an Outcome of Prolonged Learning 


Expertise is the outcome of a prolonged period of learning (Helton, 2009b). 
This definition provides a solution to the challenges posed by a statistical 
definition of exceptional performance or one based on social voting 
mechanisms. A prolonged learning outcome definition rules out innate skills 
or single-trial learning skills, as these are not likely candidates for expertise. 
Do other animal have skills that improve with long periods of time, long 
perhaps relative to their lifespan? Dogs are again a good example; they 
compete in a number of athletic activities or sports. An investigation of these 
sports provides direct evidence of long periods of skill improvement (Helton, 
2009b). A case in point is the sport of weight-pulling. Readers familiar with 
the book Call of the Wild should be familiar with weight-pulling. In the sport 
of weight-pulling a dog is harnessed to a heavy sled or cart and is tasked with 
pulling the weighted vehicle over a relatively short-distance track. The goal 
of the competition is to see which dog can pull the most weight and the dogs 
are sorted into weight classes. Weight-pulling is the canine equivalent of the 
human sports of powerlifting or weightlifting. Research does demonstrate 
breed body type or muscular skeletal morphology plays some role in weight- 
pulling performance (Helton, 2011b). There is a genetic contribution. 
Generally mastiff dog breeds are able to pull more weight for their 
bodyweight than husky breeds. Nevertheless, even in this relatively simple 
sport, dogs appear to improve markedly with practice and repeated exposure. 
To provide a case in point, Bridger, a Swiss mountain dog who was a weight- 
pulling champion for his weight class in the United States, went from an 


initial pull of 2800 lb in his first public competition to a pull of 4010 lb a 


little over a year later. Considering these dogs live at most ten or so years, 
this represented approximately 10 percent of Bridger’s likely lifespan for a 43 
percent improvement in performance. For Bridger this was a prolonged 
period of skill development which resulted in marked performance gains. 
This definition would require consideration of the lifespan of the animal 
itself. In people, ten years is often used as a rough rule of thumb and this 
would translate into roughly 10 percent or more of a person’s lifespan. Many 
animals do not live ten years. So this could be scaled to their lifespan. Some 
animals may live so briefly that expertise is no longer meaningful when 
applied to them. This would encourage integrating the expertise literature 
emerging from cognitive psychology with the life history literature emerging 


from ecology: a cognitive ecology of expertise. 


Benefits of Examining Animal Expertise 


Recognizing that other animals have expertise or are capable of developing 
expertise provides three benefits. First, animals may be studied in a manner 
untenable with people and this opens new methods to examine expertise and 
expertise development. Second, examining animals may provide an 
evolutionary understanding of the emergence of expertise. This raises central 
and interesting scientific questions. Third, recognizing animal expertise may 
have practical benefits, regarding the way we employ non-human workers, 


such as working dogs. 


New Methods 


Animals may be studied legally in a manner untenable with people. We can, 
for example, control both the genetics (breeding) and early life experiences of 
animals in ways that are unthinkable with people. This may open the path to a 
better understanding of the classic nature—nurture or talent—practice debate 
which remains controversial in the literature on people (Helton, 2008). For 
example, some recent work on dogs suggests attributions of inherent talent 
for trainability may have something to do with genetically influenced 
properties, but they may not be the cognitive traits (or talents) most people 
believe are being selected. Instead the selection appears more likely to be for 


physical shape (Helton, 2010). Alternatively, genetic selection may influence 


elements of physical and sensory capacities and these can have impacts on 
skill development in ways which are not always immediately obvious. 

While other animals could be examined, dogs in particular provide a 
useful model for understanding the impact of genetics on expertise 
development because we have selected for numerous breeds. Consider 
athletic performance such as running speed; all things equal, dogs with longer 


legs will run faster than dogs with shorter legs (Helton, 2007). Greyhounds, 


whippets, and other sight-hounds demonstrate selection for these 
characteristics and these are the breeds of dogs a person encounters at a race 
track. Alternatively, dogs for which wrestling and fighting ability have been 
selected look markedly different, for example, pit-bulls. Indeed, anatomical 
examinations comparing greyhounds and pit-bulls show clear differences and 
they have significant genetic components (Kemp, Bachus, Nairn, & Carrier, 


2005). For physical based skills the evidence based on dog breeding suggests 


that genetics is a significant factor in dog physical skill and thus, later 
expertise development. 

On the other hand, there is surprisingly, despite widespread belief, little 
research suggesting breed differences in actual cognitive abilities. Although 
people strongly believe breeds differ in intelligence and cognitive ability 
(Coren, 1994), when controlled tests are made that account for physical 
differences amongst breeds there is apparently little difference amongst 
breeds. As Scott and Fuller (1965, p. 258) concluded after a series of 
cognitive tests conducted on basenjis, beagles, cocker spaniels, fox terriers, 
and Shetland sheepdogs, “we can conclude that all breeds show about the 
same average level of performance in problem solving, provided they can be 
adequately motivated, provided physical differences and handicaps do not 
affect the tests, and provided interfering emotional reactions such as fear can 
be eliminated.” 

Indeed, our own work has found that differences in perceived 
trainability amongst breeds may have more to do with physical characteristics 


of dogs than their mental characteristics (Helton, 2010; Helton & Helton, 


2010). To summarize our findings, dog breeds perceived to be intelligent are 
typically not too small and not too big. They are not morphologically 
specialized. They are not too wide, like fighting dogs, and not too narrow, 
like running dogs; they are physically just right for cooperative tasks with 
most people. In simplest terms, breeds of dogs considered smart or easy to 
train tend to be dogs that have their heads at a height that nicely meets where 
a typical human hand hangs (the primary instrument for reinforcement and 
punishment). They are easy to train, because they are the right size for 
people. What this may imply about judgments of people’s capacities for 


expertise development is an open and interesting question. 


Evolutionary Considerations 


A cross-species investigation of expertise and its development in animals 
may help us better understand the evolution of expertise and other cognitive 
phenomena, for example, language and social cooperation. Taking animal 
expertise seriously puts expertise in evolutionary and ecological context. 
How does language relate to expertise and expertise development? Did 
language evolve to facilitate expertise development; is language necessary for 
expertise? How does expertise relate to life history development? 

Consider the role of language in expertise development. Some 
researchers have even proposed a difference between a communications 
based form of expertise, labeled interactional expertise, and more embodied 


practice-based expertise (Collins, 2004; Schilhab, 2013). Interactional 


expertise enables someone the ability to talk about a domain to others who 
are also interactional experts, who themselves have either just interactional 
expertise or interactional plus practical expertise in the domain. For example, 
a person may know all the words associated with a domain of expertise, let’s 
say surfing. She may be able to describe in detail the different wave breaks — 
point, beach, and reef — but she may not be able to actually surf. In other 
words she is an expert at talking about surfing, not actually surfing. Is this 
form of expertise evolutionarily important? If most survival skills can be 
acquired without the need for language, then why did people develop 
language (Pinker, 2010)? Is interactional expertise (the ability to talk about 
one’s domain) a means to compete socially, a means to advertise one’s 
abilities without having to risk injury by demonstrating those abilities? The 


talking surfing expert may be able to impress potential mates with knowledge 


about surfing without having to risk splitting her head on a real reef or 
drowning. 

How much does language really facilitate, or in the evolutionary past 
facilitated, skill acquisition in people? While people can undoubtedly learn 
skill relevant information from reading a book and instruction undoubtedly 
accelerates skill acquisition in modern settings (definitely facilitates 
interactional expertise), is it as useful as we believe? If other animals can 
develop the abilities to accomplish remarkable skills without language, then 
what does language really do for us? There is evidence of instruction and 
skill scaffolding amongst non-language using animals. There are a number of 
predatory mammal species which appear to take some efforts to skill up their 
offspring. Cheetah mothers capture live prey and release it for their offspring 
to practice on. They will even disable, but not kill, the prey if their kits are 
unable to handle the prey (Caro, 1994). Meerkats will de-tail a scorpion, but 
not kill it, and release it in front of their young (Thornton & McAuliffe, 
2006). Orca mothers actively encourage their young to beach themselves in 
the hunt for unsuspecting seals (Guinet, 1991). If these skills can be imparted 
without language, it raises questions regarding the evolution of language and 
the role of human-like language in expertise itself. 

Animal expertise also enables discussion of the role of skill in life 
history development. Interestingly, at least in some animals, the animals do 
not reach peak performance for evolutionarily relevant skills (for example, 
foraging skills) prior to reaching reproductive age (Forslund & Part, 1995; 
Patterson, Krzyszczyk, & Mann, 2016). Instead the animals reach peak level 
performance in mid-life, well after reaching physical maturity. Indeed, across 
many species, if you control for their natural lifespan, there is remarkable 


similarity in the pattern of peak skill development, including human hunter- 


gatherers (Helton, 2008). Interestingly, if language serves a special role in 
accelerating skill transmission and development in people, currently there is 
no evidence of this for ecologically relevant skills (like foraging). These 
issues can only be investigated if animal expertise is recognized as something 


continuous with human expertise. 


Practical Benefits 


The recognition of expertise and its development in animals may have 
practical implications. The recognition that skills require time to develop in 
working animals and that working animals share similarities with human 
experts may improve their societal use and deployment. Similar to human 
experts, working animal experts may need tools to enhance their work and 
their work may be facilitated by taking an ergonomic perspective (Helton, 
2009a). In addition, recognition of their expertise may alter the discourse on 
their use relative to other potential replacement technologies. 

There is often a naive belief that working animals perform entirely 
innate skills. No human worker would be expected to either get the skill or 
not, but would be allowed maturation and skill development. Because human 
workers can vocalize their efforts, they are able to express their motivation to 
continue with training. This is not possible with non-language using animals. 
But understanding that just like human experts, working animals need time to 
develop their skills and just as importantly time to retain those skills is 
important. Indeed methods used to gauge skill process changes in animals 
(Helton, 201 1a) may later be useful for assessing skill processes in people as 
well (Carson, Collins, & Richards, 2014). 


Conclusion 


Some animals appear to meet the definition of expertise regardless of what 
definition is offered. Unless advocating a definition of expertise that 
specifically excludes animals, for example one based on presumptions of 
consciousness, many animals are obviously experts. Recognizing animal 
expertise opens new directions for research on human expertise and itself 
raises fascinating topics. 

An obvious example is the debate about the relative role of innate ability 
(talent or genetics) and experience (training) in human _ expertise 
development. While comparative work cannot specifically answer the role of 
genetics and experience in human skill development and attainment of 
expertise (species do differ), the comparative work can be generally 
informative. Indeed a close examination of animals may at the minimum raise 
interesting questions regarding the impact of genetics and experience in 
regard to people. If dog breeds do not actually differ in cognitive abilities 
(this remains a debated issue), when physical abilities are controlled for, then 
the question would be why? Wouldn’t people have tried to select for dogs 
which are cognitively gifted for some problem or skill? Does neuro-plasticity 
mean this kind of selection is inherently challenging or that people cannot 
help but instead be influenced in their judgments by external morphological 
characteristics, in other words, appearances? 

Recognizing animal expertise also means society might take more 
interest in the use of other animals in vitally important occupations. Medical 


science, for example, is an arena where expert dogs or other animals could 


emerge as a useful addition to current diagnostic science. Dogs, for example, 
show promise in this area, but the skills would require extensive training 
(Helton, 2009b). There would be no reason to presume the examination of 
novice or unskilled dogs would be particularly informative and this is what 
often happens in examinations of these areas. To use an analogy, are people 
capable of back-flips? If I took a random sample of non-gymnasts and tested 
them, perhaps it would look like people are not capable of acquiring the 
back-flip skill. This is often the approach taken with working dogs; society 
does not recognize that there is expertise in animals and like us, expertise 


takes a long time to develop. 
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Introduction 


The study of expertise has a long history (see Ericsson, Chapter 1, this 
volume). In the first part of this chapter we emphasize a period of research 
roughly from the mid-1950s into the 1980s when empirical laboratory studies 
of expert reasoning were first combined with theoretical models of human 
thought processes that could reproduce the observable performance. In the 
second part of this chapter we will characterize some of the enduring insights 
about mechanisms and aspects of expertise that generalize across domains, 
reflecting on the original theoretical accounts but also considering more 


recent ones. 


The Historical Development of Expertise 
Studies 


While there was earlier important work related to scientific studies of 


complex thinking and expertise (e.g. Boden, 2006), in the period of focus 


(1950s—1980s) a number of trends came together to provide enough traction 
for the field of expertise studies to “take off.” There were three main roots to 
this impetus: artificial intelligence, cognitive psychology, and education. We 
will overview these three roots briefly. If, to the reader, it may be confusing 
at times whether we are addressing artificial intelligence, cognitive 
psychology, cognitive science, or even education, the fact of the matter is that 
during a crucial long span of time in the development of expertise studies, it 


was hard to tell the difference. 


Artificial Intelligence: Expertise in the Code 


Early computer models developed by Herbert Simon and Allen Newell 
demonstrated that it is relatively easy for computational devices to do some 
things worthy of being considered intelligent. This breakthrough at Carnegie 
Mellon University (then Carnegie Institute of Technology) was based on the 
confluence of two key realizations that emerged from the intellectual milieu 
that was emerging between Carnegie and Rand at the time, curiously set 
within the context of a new business school (Augier & March, 2011; Augier 
& Prietula, 2009). 


Allen Newell, Clifford Shaw, and Herbert Simon envisioned that 


computers could be used to process “symbols and symbol structures.” To 
explore this, they developed what was to become the first list-processing 
computer language, namely the Information Processing Language (IPL), 
which enabled them to build computer programs with arbitrarily complex list 
structures, and manipulate them recursively. Second, they incorporated the 
concept of levels of abstraction in articulating their theories and, 
consequently, in developing their programs. These developments allowed 
them to address two critical technical problems in articulating theories and 
models of thought. The “specification problem” is how to specify 
components and processes sufficient to capture the characteristics of interest. 
The “realization problem” is to implement the specification in an actual 
physical system to enable synthesis (Newell & Simon, 1956). The view of 
humans and machines as complex information processing systems had been 


formulated. 


During these early years, the first artificial intelligence (AI) program, 
called the Logic Theorist (LT; Newell & Simon, 1956), was written. LT was 
initially run on the JOHNNIAC computer at the RAND Corporation in Santa 
Monica, California. Significantly, it was able to prove theorems in predicate 
calculus in a manner that mimicked human adults (Newell & Simon, 1972). 
Of particular relevance to expertise, LT was able to create some novel proofs. 
In fact, a turning point in the development of AI was the presentation of their 
Logic Theorist paper in 1956 at the Dartmouth Summer Research Project on 
Artificial Intelligence (McCorduck, 1979). Later, the heuristics from LT were 
generalized into a model that could solve problems in many different 
domains, the General Problem Solver (GPS; Ernst & Newell, 1969). In 
addition, there were also other computer models that were built not as 
simulations of human problem-solving, but based on effective computation 
designed to represent artificial methods for producing intelligent action. For 
example, Samuel’s (1959) checkers-playing program was able to challenge 
and beat excellent human checkers players. 

Consequently, it was demonstrated that computer programs could model 
the human problem-solving processes (form and function). This was an 
important leap for psychological theories in general, and expertise studies in 
particular, as such approaches afforded the possibility of asserting 
hypothetical constructs in a way that captured what those constructs 
contributed to performance. Furthermore, the computational forms 
themselves became instantiations of the theoretical apparatus that accounted 
for such performance, providing explanatory representations of both the 
process and product of thought — the theory was in the code (Prietula, 2011). 

Such “cognitive models” often provided a map or trace of the construct 


dynamics relevant to the particular task at hand, which could then be 


compared to an individual’s performance on the same task, often using verbal 
protocol analysis (e.g. Newell, Shaw, & Simon, 1958). Work of this era led to 
a variety of simulations in the psychological sciences (Borko, 1962; Dutton 
& Starbuck, 1971). In fact, perhaps one of the most influential and 
comprehensive publications describing this early work was Feigenbaum and 


Feldman’s anthology, Computers and Thought (1963), containing an array of 


articles by psychologists, computer scientists, linguists, management 
scientists, and philosophers. However, applications constructed to explicitly 
study human expertise came later, building on these early successes. 
Examples include models of skill acquisition as general cognitive 
architectures (e.g. Anderson, 1982; Anderson & Lebiere, 2003; Newell, 
1990) and cognitive models of specific task performance variations (e.g. 
Johnson et al., 1981). Psychology has since embraced computational models 
in general as a method to examine a wide range of psychological phenomena 
(Sun, 2008). However, it was these early successes that spawned themes 


regarding expertise pertinent to the present chapter. 


Cognitive Psychology: Expertise in the Experts 


In the 1950s—1960s, a new generation of researchers who were unabashedly 
interested in cognition moved beyond the animal models of the behaviorists, 
engaging experimental methods and theoretical constructs addressing more 
complex phenomena of higher reasoning in humans such as planning (Miller, 


Galanter, & Pribram, 1960), thinking (Bartlett, 1958), concept formation 


(Bruner, Goodnow, & Austin, 1956), and mental structures and their 


functioning (Miller, 1956). At the same time, the development of the 


information processing viewpoint in psychology was led by George Miller as 
well as Simon and Newell, who not only asserted higher level theoretical 
constructs for problem-solving, but also asserted that “an explanation of an 
observed behavior of the organism is provided by a program of primitive 
information processes that generates this behavior” (Newell et al., 1958, 
p. 151), situating their prior work on the Logic Theory Machine (Newell & 
Simon, 1956) within their information processing framework. Support and 
recognition for their methodology was appreciated, and even embraced, by 


the discipline (e.g. Hovland, 1960), and the general idea that components of 


complex human reasoning could be studied was a key impetus for the study 
of expertise. 

As we noted, the idea that computation could exhibit intelligent behavior 
reinforced the growing idea that computers and their programs could stand as 
formal models of human cognition. This grew into a pervasive stance toward 
human and machine cognition, the information processing model. Cognitive 
Psychology and Computer Science merged into a very close collaboration 


(along with linguistics and a few other fields) that was named Cognitive 


Science. The Cognitive Science Society was founded in 1979, but its 
multidisciplinary foundations were asserted earlier in what was to become its 
outlet journal, Cognitive Science, in 1977. In that first issue, Alan Collins 
asserted that cognitive science was “defined principally by the set of 


problems it addresses and the set of tools it uses” (Collins, 1977, p. 1). As a 


matter of fact, that first issue covered four topics: a computational knowledge 
representation language, a computational question-answering system, an 
empirical examination of semantic memory, and a discussion outlining a 
computational frame-based knowledge _ representation. And _ these 
computational models and theories provided alternatives to the behaviorist 
(stimulus-response, no internal mental mechanisms) theories that had 
dominated psychology for the prior half a century (more on this in our 
treatment of psychology and expertise below). Newell and Simon, two 
pioneers of the information processing model, asserted this forcefully. It 
should be noted that Newell, an electrical engineer, and Simon, an economist 


and polymath, believed they were doing psychology 


As far as the great debates about the empty organism, behaviorism, 
intervening variables, and hypothetical constructs are concerned, we 
take these simply as a phase in the development of psychology. Our 
theory posits internal mechanisms of great extent and complexity, and 
endeavors to make contact between them and the visible evidences of 


problem solving. That is all there is to it. 


(Newell & Simon, 1972, pp. 9-10) 


This has to do with alternative basic approaches to achieving 


intelligence in a computational device, what have been termed “weak and 


strong methods” (Newell, 1969, 1990). The earliest successful AI programs 
utilized weak reasoning and problem-solving methods that were drawing on 
descriptions of human thought processes. Weak methods are highly portable, 
generalizable methods of reasoning that do not depend on the particular 
content of the domain of problem-solving but, in being so, are less capable of 
finding solutions without extensive search. One example is “generate and 
test” (generate known next steps, and test to see if any of them yields success, 
else go back to generate, from the point of each newly generated state). 


Another example is “means—ends analysis”: 
e Represent the ending goal state, what you are trying to achieve; 
e Represent where your progress has brought you right now; and 


e Try to find some currently available computational operator as a 
means that can decrease some aspect of the difference between these; 


or 
e Try to represent and solve sub-goals to bridge the gap. 


Strong methods are more heavily dependent on rich knowledge of the 
problem-solving area, and on understanding which operators are likely to be 
successful in encountered specific situations. They are domain specialists, not 
generalists. Consequently, there is a trade-off between the power of general 
search and the capabilities of task-relevant knowledge — more task-relevant 
knowledge reduces the need for search. Make the move-generator smarter; 
make the goals task-specific. 

When early AI was being applied in relatively simple and well- 
structured areas, such as simple games like Samuel’s checkers program, weak 


methods fared fairly well. As the field developed and researchers started to 


address richer, complex, and knowledge-laden task environments, such as 
medicine (Pauker, Gorry, Kassirer, & Schwartz, 1976; Shortliffe, 1976) and 


chemical spectral analysis (Buchanan & Feigenbaum, 1978), the need for 


ever-stronger methods became clear. Portability across task domains had to 
be sacrificed in favor of capability, but narrowly restricted capability. Over 
time, the development of computational models that did not aim at explicit 
representations of general psychological mechanisms emerged, but focused 
on capturing the methods and forms of exceptional performance on a problem 
in a particular task domain — expert systems (Buchanan, Davis, & 
Feigenbaum, 2006; Buchanan, Davis, Smith, & Feigenbaum, Chapter 7, this 
volume). Although the form and function of expert systems varied, the 
common goal was generally to “exhibit some of the characteristics of 
expertise in human problem-solving, most notably high levels of 
performance” (Buchanan et al., 2006, p. 87). Toward this end, the 
construction of these systems often relied on discerning the knowledge-based 
methods of human domain experts, and using that to inform an engineering 
process that yielded a coherent, functioning system capable of comparable or 
better performance. One extremely useful method brought to bear in 
understanding performance in general (for cognitive models) and skill in 
particular (for discerning expertise) was the method of instructing 
participants to “think aloud” (see Ericsson & Simon, 1993, for a description 
of a wide range of different instructions used to elicit thinking aloud). 

The use of “think-aloud” instructions is perhaps best exemplified early 
on by Duncker’s (1945) classic work on general problem-solving. Duncker 
(1945) took detailed notes on different thoughts generated by participants 
while they were thinking aloud while solving challenging problems that were 


drawn from everyday life. Later, during their research using think-aloud 


instructions to study logical reasoning, problem-solving, and chess playing, 
Newell and Simon (1972) discovered research that had used this method to 
study chess expertise by Adrian de Groot in a Dutch dissertation from 1946, 
which was translated into English (de Groot, 1946, 1965), Subsequent work 
by Ericsson and Simon (1980, 1993) refined this methodology for “think 
aloud” and developed standardized instructions based on theoretical analyses 
of the verbalization processes of thoughts and proposed rigorous analyses 
based on transcriptions of the participants’ thoughts based on tape recordings 
of the sessions. When participants were instructed to remain focused on the 
task and merely verbalize their thoughts (as opposed to explaining their 
thoughts), reviews (Ericsson, Chapter 12, this volume; Ericsson & Simon, 
1980, 1993; Fox, Ericsson, & Best, 2011) found no evidence for differences 
in performance accuracy between participants thinking aloud versus solving 
the same problems in silence. 

Not surprisingly, groundbreaking progress in this regard came from the 
information processing camp in their studies of problem-solving that used 


this type of analysis (Newell & Simon, 1972), especially in their studies 


(following de Groot) of expertise in chess (Chase & Simon, 1973a, 19735). 
This research demonstrated striking differences between experts and novices 
in their ability to perceive relevant information and their ability to think about 
and solve problems in their domain of expertise, such as chess (Charness, 
1976, 1979, 1981; Chi, 1978), physics (Chi, Feltovich, & Glaser, 1981; 
Larkin, McDermott, Simon, & Simon, 1980), and medicine (Elstein, 
Shulman, & Sprafka, 1978). Furthermore, this (thinking aloud procedure) 
was to serve as a foundational method for some of the earliest computer 


programs, such as the GPS mentioned in the previous section: 


Working from the “protocol” recording the behavior of a test person 
solving a logic problem, a computer program called GPS (for General 
Problem Solver) is developed, which leads to a psychological theory of 
human problem solving. It is shown, how data giving the same results as 
derived from the protocol yields to an analysis in terms of a program 


characterized by a recursive structure of goals and subgoals. 


(Newell & Simon, 1961, p. 109) 


It is interesting to think about whether a field of expertise studies could have 
emerged at all, and if so, what it could possibly have looked like, if 
alternatives to American neo-behaviorism had not emerged. Clark Hull’s 
theory of linked habits was essentially an elaboration of Thorndike’s 
behaviorism, but marshaling quantitative models to discern habit strength. B. 
F, Skinner disavowed causal explanation involving mental mechanisms, such 
as thoughts and memories, asserting that cognitive theories of learning are 
unnecessary (Hunt, 1993). Consequently, would we have discovered that 
experts do not just complete tasks and solve problems faster and better than 
novices, but often attain their solutions in qualitatively different ways? 
Would we have discovered that there are fundamental representational 
differences in how they see the problems? Would we have discovered that 
experts frequently spend a greater proportion of their time in initial problem 


evaluation compared to novices (e.g. Glaser & Chi, 1988, regarding “Experts 


spend a great deal of time analyzing a problem qualitatively”; Lesgold et al., 
1988; see also Kellogg, Chapter 23, this volume, on planning by professional 
writers, and Noice & Noice, 2006, on the deep encoding by professional 


actors as they study their lines)? 


Similar questions can be posited regarding the development of the 
cognitive science movement itself, which was highly influenced by the rise of 
the general-purpose computer, allowing programs to function as “mid-range” 
theoretical constructs in cognition mediating observable behavior (e.g. 
attention, plans, goals, concepts, aspiration levels, learning, and search) and 
“low-level” engineering efforts to mathematically model neuronal cell 
assemblies (e.g. Ashby, 1952; Hebb, 1949; McCulloch & Pitts, 1943; von 
Neumann, 1958). The rise of intermediate-level (i.e. above machine or 
assembler code), symbolic programming languages permitted cognitive 
science to “represent” constructs of novice and expert deliberations in forms 
amenable to discussion in the context of the concepts, functions, and 
associations (e.g. proximal, temporal) revealed empirically through 
behavioral studies. For example, both the Logic Theorist and the General 
Problem Solver were written using (albeit different versions of) a 
RAND-Carnegie Tech programming language, mentioned earlier, called IPL 
(Information Processing Language). IPL provided a necessary innovation (at 
the time) for non-numeric data types (symbols, lists) to allow important 
constructs (e.g. goals, sub-goals, generate-and-test methods, and heuristics) 
of the cognitive theory of task performance to be represented in the model 
(Ernst & Newell, 1969; Newell & Shaw, 1957). IPL was also engaged by 


other researchers, such as the early work on simulating concept formation 


(e.g. Hunt & Hovland, 1961). However, it is likely that these technological- 
methodological advances led to the emergence of the influential information 


processing approach in cognitive science (Newell & Simon, 1972; Reitman, 


1965), which was itself to lay the foundations for expertise studies. Not every 
researcher could, or had to, build a computational model, but the successful 


construction of these computer models provided evidence of sufficiency in 


support of various cognitive constructs. 

While modern views of expertise retain a criterion of superior 
(observable, repeatable) performance, as did behaviorism, there is also 
considerable interest and theorizing about mediating processes and structures 
that support and can be developed to produce these superior performances. 
Interestingly, current theorizing about the critical role of deliberate practice in 
the development of expertise embodies characteristics reminiscent of these 
earlier, neo-behaviorist approaches, such as the need for clear goals, repeated 
practice experiences, and the vital role of feedback about the quality of 
attempts (Ericsson, Chapter 38, this volume; Ericsson, Krampe, & Tesch- 
R6mer, 1993). These insights into effective learning had an important impact 


on efforts to improve education and training. 


Educational Psychology and Instructional Design: From Novice to 
Expert 
The emergence of the cognitive science perspective in psychology also 
impacted educational psychology, with studies of expertise taking on a new 
role. Expert cognition was conceived as the “goal state” for education, the 
criterion for what the successful educational process should produce, as well 
as a measure by which to assess its progress, serving to inform pedagogical 


design and teacher evaluation (e.g. Berliner, 1988; Feldon, 2006). In this 


regard, advanced methods have now been developed for eliciting and 
representing the knowledge of experts (see Lintern, Moon, Klein, & 
Hoffman, Chapter 11, this volume). Novice cognition (as well as that of 
various levels of intermediates) could serve as “initial states,” as models of 
the starting place for the educational process. In a sort of means—ends 
analysis, the job of education was to determine the kinds of operations that 
could transform the initial conditions into the desired more expert-like ones 


(Glaser, 1976). Although it is tempting to believe that upon knowing how the 


expert does something, one might be able to “teach” this to novices directly, 


this has not been the case (e.g. Klein & Hoffman, 1993); the achievement of 
expertise is the product of a long, complex process. 

Expertise (as we are defining it) is a long-term developmental and 
adaptive process, resulting from rich instrumental experiences in the world 
and extensive and deliberate practice and feedback. (A current challenge for 
pertinent education and training is whether and how this experiential process, 
and the opportunity it provides for practice and feedback, can be compacted 


or accelerated, Hoffman et al., 2014). However, the kinds of experiences, 


practice, and feedback that are necessary depend on the characteristics of the 
task environment, and how individuals adapt to those characteristics within 
the constraints of their cognitive limitations. Simon’s early descriptions of 
bounded rationality and the task environment are perhaps best reflected in the 


oft-quoted The Sciences of the Artificial (Simon, 1969, p. 25): 


A man, viewed as a behaving system, is quite simple. The apparent 
complexity of his behavior over time is largely a reflection of the 


complexity of the environment in which he finds himself. 


Though perhaps an oversimplification, the statement does emphasize the 
important role of the task environment is shaping the observable behavior. 
This important role of the task environment was best articulated and 
exemplified by the detailed analyses, conducted by Newell and Simon, of 
task environments involved in human and computational problem-solving 
processes (Newell & Simon, 1972). Later, Simon (1990) would invoke a 
metaphor that would also serve to infuse the importance of the task 
environment into the theoretical apparatus of other theorists, noting “Human 
rational behavior (and the rational behavior of all physical symbol systems) is 
shaped by a scissors whose two blades are the structure of the task 
environments and the computational capabilities of the actor” (Simon, 1990, 
p. 7). The shaping of that behavior toward expertise over time, the 
developmental or educational process leading toward expertise, was a critical 
missing component of explaining expertise. 

Consequently, some early expert—novice difference research led directly 
to the creation of new methods of instruction. This is particularly true in 


medical education where early expert—novice studies (Barrows, Feightner, 


Neufeld, & Norman, 1978; Elstein et al., 1978) led to the creation of 
“problem-based learning” (Barrows & Tamblyn, 1980). Over a long period of 
time, problem-based learning (and variants) has come to pervade medical 
education, as well as making significant inroads into all types of education, 
including K-12, university, and every sort of professional education (see 
Ward, Williams, & Hancock, 2006, for a review of the use of simulation in 
training). 

Ongoing, and more recent, research in education in a particular domain 
at high levels of performance often involves attempts at explicating expertise 
in that domain to inform the design of the educational experiences in their 
task environments, such as emergency medicine (Pelaccia et al., 2016; Wears 
& Schuber, 2016), skills and standards for teachers (Anthony, Hunter, & 
Hunter, 2015; Kaub, Karbach, Spinath, & Brunken, 2016), and designing 
human-machine systems (Yu, Honda, Sharqawy, & Yang, 2016). Relatedly, 


the emergence of a focus on studying how individuals and groups make 
decisions in real-world task environments (as opposed to replicating elements 
of the task environment in a controlled context), generally referred to as 
“naturalistic decision making” (see Klein, 2008; Mosier, Fischer, Hoffman, & 
Klein, Chapter 25, this volume) has led to insights and innovations regarding 
training and education based on expertise (Keller, Cokely, Katsikopoulos, & 
Wegwarth, 2010; Klein, 2016; Klein, Woods, Klein, & Perry, 2016). We now 
briefly recapitulate historical roots discussed so far and offer some follow-on 


developments. 


Recapitulation and Extensions 


The modern study of expertise started with analyses of expert chess playing 
and other types of games, and domains with formal structures and rules, as 
we have already described. This was followed by extensions into more 
knowledge-intensive fields, emphasizing the critical role of knowledge, 
knowledge organization, knowledge access, and so forth. In an influential 
book, Bloom (1985) reported how individuals attained an international level 
of performance in six very different domains. In addition, the first conference 
explicitly using the word “expertise” in its title was focused primarily on 


domains of expertise where knowledge is critical, including physics, 


medicine, and computer programming (Chi, Glaser, & Farr, 1988), but also 
research on high levels of practical skills. A subsequent conference attempted 
to broaden the evidence to include other domains of expertise, such as sports, 
music, writing, and decision making (Ericsson & Smith, 1991a). 

This was followed by several edited and authored books on topics 
including the relations of the psychology of expertise to the field of Artificial 
Intelligence (Hoffman, 1992), sports and motor expertise (Starkes & Allard, 


1993; Starkes & Ericsson, 2003), as well as covering a wide range of research 


on human and computer expertise (Ericsson, 1996; Ericsson & Smith, 1991a; 
Feltovich, Ford, & Hoffman, 1997). The first edition of this handbook 
appeared in 2006 (Ericsson, Charness, Hoffman, & Feltovich, 2006) which 


integrated what was known about the structure and acquisition of expertise 


and expert performance, and books were written about particular domains, 
such as sports (Baker & Farrow, 2015; Farrow & Baker, 2013). General 


books on the topics of expertise and expert performance have been published, 


focusing on professional development (Ericsson, 2009), accelerating the 


development of expertise (Hoffman et al., 2014), as noted earlier, and 
expertise in professional decision making (Hoffman, 2007). 

One thing these contributions bring out is the deep entanglement of 
expertise studies with the history and evolution of basic and applied cognitive 
science, broadly, across the 1980s and 1990s (see Hoffman & Deffenbacher, 
1992; Hoffman & Militello, 2008). This brings us to the second major section 


of this chapter. 


Toward Generalizable Characteristics of 
Expertise and Expert Performance 


We now attempt to crystallize the enduring findings from the study of 
expertise. We will draw upon generalizable characteristics of expertise 


identified in earlier reviews (Glaser & Chi, 1988) and describe how these 


characteristics have been refined and developed in light of studies of 
reproducibly superior performance. In doing so we contrast two general 
approaches to expertise studies, what we call the “expert—novice” and “expert 
performance” approaches. The approaches differ most in how people are 
selected to be studied at various levels of relative expertise. In the, mostly 
earlier, expert-novice tradition, experts (and novices and_ various 
intermediates) were identified by such factors as experience and educational 
levels. In the expert performance scheme, experts (and novices, etc.) are 
selected for their relative superior performance on representative tasks from 
their domain. We will also discuss the original theoretical accounts for the 
findings presented, as well as more recent findings and theoretical treatments 


reviewed in the chapters of this handbook. 


Expertise is Limited to a Domain of Knowledge, and Elite 
Performance is Mediated by Domain-Specific Skills and 
Adaptations 
In their pioneering review Glaser and Chi (1988, p. xvii) stated that the first 
characteristic of expertise is that “Experts excel mainly in their own domain.” 
They argued that the superiority of experts could be related to their 
organized, relevant knowledge rather than some global superiority, such as 
intelligence or better “reasoning.” (Recall our earlier discussion of “strong 
methods” in artificial intelligence.) They cited Voss and Post’s (1988) 
research, in the expert—novice tradition, on problem-solving in political 
science by experts and novices. Novices, such as college students, were 
thinking about the presented problems at a very concrete level, whereas the 
experts (their professors) thought about the same problems in more abstract 
ways. Glaser and Chi (1988) also cited earlier research on taxi drivers’ 
knowledge about how to take a passenger between two points in a city and 
found that the more experienced drivers were able to generate a larger 
number of possible routes (Chase, 1983). Similarly they pointed to expert 
physicians’ more differentiated knowledge of diseases into numerous more 
specific disease variants (Johnson et al., 1981). Several of the other 
characteristics differentiating more experienced and knowledgeable 
individuals (experts) from less experienced individuals (novices) concerned 
the experts’ “larger patterns” and “deeper (more principled)” encoding of 

domain-related information. 
A different approach to studying expertise was introduced by Ericsson 
and Smith (1991b). According to this approach, the expert performance 


approach, the focus should not be on identifying experts based on their more 


extensive training and professional experience as had been done in much 
prior research, but rather by their reproducibly superior performance on 
representative tasks that capture expertise in a domain. This approach 
attempts to identify individuals with consistently superior performance, such 
as chess players who consistently win tournament games in chess, surgeons 
with superior outcomes for their patients, and musicians who consistently win 
competitions. The criterion for expert posed by Bloom (1985) was that they 
had to have won an international competition in swimming, tennis, music, 
mathematics, or neuroscience. When we apply this definition of expert 
performance, it is appropriate to ask if it is possible to attain the highest level 
of performance in several domains. When we restrict the claims to 
individuals who can perform successfully and win at an international level in 
an established well-known domain, then it is clear that people hardly ever 
reach this highest level in more than a single domain of activity (Ericsson & 
Lehmann, 1996). An expert in one domain, such as ballet, cannot, without 
extensive training, reach a similar level in another different domain, such as 
tennis. 

There was an interchange on the role of general abilities and multiple 
intelligences in acquiring expertise between Ericsson and Charness (1994) 
and Gardner (1995; see also Ericsson & Charness, 1995). More recently, 
Ericsson (2014) showed in a review that general abilities, such as IQ, are 
correlated with performance for beginners, but with increased performance 
attained after years of practice, the relation between performance of experts 
and their general abilities was no longer significant. Ericsson (2014) 
proposed that new, cognitive structures and skills were acquired with training 
that started mediating performance and eventually would fully mediate the 


performance of experts. 


There is a large body of evidence supporting the need to acquire 
adaptations and skills to attain expert levels of performance. Simon and 
Chase (1973) showed evidence that it would take a decade or more of playing 
chess for a chess player to start winning consistently at the international 
tournaments and, thus, to reach the status of grand master. Similar findings 
for a wide range of domains were described and reviewed by Ericsson et al. 
(1993) (for a more complete recent summary see Ericsson, Chapter 38, this 
volume). There have been numerous claims that exceptional individuals are 
gifted and thus are able to attain a truly exceptional performance by adult 


standards with essentially no prior practice (Epstein, 2013; Gagné, 2013), but 


see alternative accounts of the same reported phenomena that do not require 
assumptions of gifts (Ericsson & Pool, 2016; Ericsson, Roring, & 
Nandagopal, 2007). 

The “task specificity” that is asserted here (limitation of expertise to a 
focus domain), is also characteristic of expertise in perceptual-motor skills 
(e.g. Fitts & Posner, 1967; Rosenbaum, Augustyn, Cohen, & Jax, 2006) and 
is exemplified in many chapters in this new edition of the handbook, but in 
particular in perceptual diagnosis and surgery (Norman et al., Chapter 19), 
sports (Williams, Ford, Hodges, & Ward, Chapter 34), and music (Lehmann, 
Gruber, & Kopiez, Chapter 28). 


The Intertwining of Knowledge and Basic Reasoning in Expertise 


During the late 1960s and the 1970s, maintaining a traditional distinction 
between domain-specific knowledge and skills, and general reasoning 
abilities, was becoming less tenable. Even the most basic elements of 
reasoning were shown to be highly intertwined with knowledge (e.g. Wason 
& Johnson-Laird, 1972). Knowledge was no longer seen as a “nuisance 
variable” in experiments, but as a dominant source of variance in many 
human tasks. In particular, Newell and Simon (1972) found that problem- 
solving and skilled performance in a given domain were primarily influenced 
by domain-specific acquired patterns and associated actions. We will first 
describe the research focusing on expertise, where individuals with extensive 
experience and knowledge related to a domain of activity are referred to as 
experts and individuals with less experience and knowledge are referred to 
novices or intermediates. Research comparing experts and novices in a given 
domain has focused on individual differences in acquired knowledge and 
how changes in the content and structure of this knowledge were associated 
with more experience in a domain and a longer period of education. After 
discussing a few general characteristics associated with this definition of 
expertise, we move to a different definition that focuses on objectively 
measurable differences in performance, namely expert performance. We will 
at that time discuss the expert performance approach toward identifying the 


mechanisms that mediate expert performance more generally. 


Differences in Expertise: The Amount and Structure of Knowledge 


As with its relationship to basic reasoning noted above, acquired knowledge 
in a domain was found to be associated with changes in fundamental types of 
cognitive processing. Chi (1978) compared experienced chess-playing 
children with other children in their performance on memory and learning 
tasks related to chess. The differences in experience, knowledge, and skill in 
chess produced differences, in favor of the young chess players, in such basic 
learning processes as the spontaneous use of memory strategies such as 
grouping and rehearsal, the ability to use such strategies even under 
experimental prompting, and the amount of information that could be held in 
short-term memory (Chi, 1978). Similarly, Voss and colleagues (Chiesi, 
Spilich, & Voss, 1979; Spilich, Vesonder, Chiesi, & Voss, 1979) extended 


this kind of research into other forms of learning. They first measured the 


amount of knowledge that individuals had acquired and then separated them 
into groups with a high and low level of knowledge. Studying high- and low- 
knowledge individuals with regard to the game of baseball, they found that 
compared to the low-knowledge individuals, high-knowledge ones exhibited 
superior learning for materials from that and only that particular domain. In 
particular, high-knowledge individuals had greater recognition and recall 
memory for new material involving game play, could make useful inferences 
from smaller amounts of partial information, and were better able to integrate 
new material within a coherent and interconnected framework (organized, for 
instance, under a common goal structure). Glaser and Chi (1988) found that 


expertise in a domain is characterized by “large meaningful patterns” (p. xvii) 


and representations that are at a “deeper (and more functional level)” (p. 


XVili). 


Expertise Involves Larger and More Integrated Cognitive Units 


With increased experience and practice, most people cognitively organize the 
perceptually available information in their working environment into larger 
units. 

In the 1960s and early 1970s, de Groot (1965) and Chase and Simon 
(1973a, 1973b) studied master level and less accomplished chess players. In 
the basic experimental task, participants were shown a chess board with 
pieces representing game positions from real games. Participants were shown 
the positions for only five seconds, and they were then asked to reproduce the 
position they had seen. After this brief glance, an expert was able to 
reproduce more pieces in the presented chess position than a novice. In the 
pioneering studies by de Groot (1946) the recall performance by world-class 
players was nearly perfect (for chessboards with 25 pieces) and worse by 
skilled players at the club level. In the famous studies by Chase and Simon 
(1973a, 1973b) the expert recalled four to five times the number of pieces 
recalled by the novice, who recalled about five pieces, or about the number of 
items that can be maintained in short-term memory by rehearsal. 

The original explanation by Chase and Simon (1973a, 1973b; Simon & 


Chase, 1973) for expert superiority involved “chunking” in perception and 


memory. With experience, experts acquire a large “vocabulary” or memory 
store of board patterns involving groups of pieces, or what were called 
chunks. A chunk is a perceptual or memory structure that bonds a number of 
more elementary units into a larger organization (e.g. the individual letters 


cH) 6 a) 


c”, “a,” and 


665,99 
rT 


into the word “car’’). When experts see a chess position from 


a real game, they are able to recognize such familiar patterns. They can then 


associate these patterns with moves stored in memory that have proven to be 
good moves in the past. Novices do not have enough exposure to game 
configurations to have developed many of these kinds of patterns. Hence they 
deal with the board in a piece-by-piece manner. Similarly, when experts are 
presented with chess boards composed of randomly placed pieces that do not 
enable the experts to take advantage of established patterns, their advantage 
over novices for random configurations amounts to only a few additional 
pieces. 

These basic phenomena attributed to chunking were replicated many 
times, in chess but also in other domains, such as GO, bridge, electronics 
diagrams, and football play sketches (see also, Ericsson, Chapter 36, this 
volume; Gobet & Charness, Chapter 31, this volume). The identified chunks 
were not only larger but also reflected a deeper meaningful structure. For 
example, one chunk of chess pieces for an expert might be a “king defense 
configuration,” composed of a number of individual chess pieces. In many 
domains experts develop abilities to encode perceptual characteristics of 
objects and scenes to facilitate rapid judgment and the generation of 


appropriate actions (Landy, Chapter 10, this volume). 


Expertise Involves Deeper and More Functional Representations of 
Tasks 


Influential findings came from the early work in physics (Chi et al., 1981) 


and medicine (Feltovich, Johnson, Moller, & Swanson, 1984; Johnson et al., 


1981). In the basic task from the physics study, problems from chapters in an 
introductory physics text were placed on individual cards. Expert (professors 
and advanced graduate students) and novice (college students after their first 
mechanics course) physics problem-solvers sorted the cards into groups of 
problems they would “solve in a similar manner.” The finding was that 
experts created groups based on the major physics principles (e.g. 
conservation and force laws) applicable in the problems’ solutions. Novice 
groupings were organized by salient objects (e.g. springs, inclined planes) 
and features contained in the problem statement itself. Similarly, in studies of 
expert and novice diagnoses within a sub-specialty of medicine, expert 
diagnosticians organized diagnostic hypotheses according to the major 
pathophysiological issue relevant in a case (e.g. constituting the “Logical 
Competitor Set” for the case; e.g. “lesions involving right-sided heart volume 
overload”), while novice hypotheses were more isolated and more dependent 
on particular patient cues. 

Similar results have been shown from yet other fields, using somewhat 
different methods that compared the performance of groups of adults who 
differ in their knowledge about a given domain. For example, Voss and co- 


workers (Spilich et al., 1979) studied ardent baseball fans and more casual 


baseball observers. Participants were presented with a colorful description of 


a half-inning of baseball and were then to recall the half-inning. Expert recall 


was structured by major goal-related sequences of the game, such as 
advancing runners, scoring runs, and preventing scoring. Novices’ recall 
contained less integral components, for example, observations about the 
weather and the crowd mood. Novice recall did not capture basic game- 
advancing, sequential activity nearly as well. More recent research on fans 
who differ in their knowledge about soccer and baseball has found that 
comprehension and memory for texts describing games from these sports is 
more influenced by relevant knowledge than by general verbal abilities (see 
also Hambrich & Engle, 2002). 

In sum, individuals with more knowledge and experience have a more 
complex and appropriate structure of their knowledge, which allows them to 


think and reason in a deeper and more functional manner. 


Mechanisms Underlying Expert Performance 


When knowledge is viewed as the primary source of difference associated 
with expertise, as was the primary focus in the expert—novice approach, it 
makes sense to study the structure of individuals’ knowledge. If, on the other 
hand, we are interested in studying the individual differences in objective 
performance that define expertise in a domain, such as winning chess games 
in tournaments, and having superior outcomes for patients after cancer 
surgery, a different approach is needed. 


In the expert performance approach to expertise (Ericsson & Smith, 


1991b; Ericsson & Ward, 2007), researchers attempt to identify those tasks 
that best capture the essence of expert performance in the corresponding 
domain, and then standardize representative tasks that can be presented to 
individuals with different amounts of experience. For example, medical 
doctors and residents differ in their ability to diagnose diseases based on X- 
rays. It is possible to present many participants with the same X-rays and ask 
them to think aloud as they diagnose the X-rays. By having experts 
repeatedly perform the diagnoses, experimenters can identify differences in 
the thought processes associated with superior accuracy in diagnosis 


(Ericsson, 2015). Hypothesized mechanisms can then be evaluated by 


designed experiments (Ericsson, Chapters 12 and 36, this volume). The 
superior performance on tasks related to the associated domain of expertise 
has been successfully described by different psychometric factors (e.g. expert 
reasoning and expert working memory) than those general ability factors that 
describe the performance of individuals with lower levels of performance, 


such as beginners and novices (Ericsson, 2014; Horn & Masunaga, 2006; and 


see Ackerman & Beier, Chapter 13, this volume, for a review of individual 


differences as function of level of expertise). 


From Short-Term to Long-Term Working Memory in Expertise 


The once-popular hypothesis that all cognitive processes, including those of 
individuals with higher levels of performance and experts, were uniformly 
constrained by a severely limited short-term memory (STM), was questioned 
in the mid-1970s. If working memory capacity could be expanded then there 
were many different possibilities for experts to improve their performance 
beyond developing larger chunks. In a dissertation supervised by William 
Chase, Charness (1976) showed that expert chess players do not rely on a 
transient short-term memory for storage of briefly presented chess positions. 
In fact, they are able to recall positions, even after the contents of their short- 
term memory have been completely disrupted by an interfering activity. 
Subsequent research has shown that chess experts have acquired memory 


skills that enable them to encode chess positions in long-term working 


memory (LTWM, Ericsson & Kintsch, 1995). The encoding and storage of 
the chess positions in LTWM allow experts to recall presented chess 
positions after disruptions of STM, as well as being able to recall multiple 
chess boards presented in rapid succession (see Ericsson, Chapter 36, this 
volume, and Gobet & Charness, Chapter 31, this volume, for an extended 
discussion of new theoretical mechanisms accounting for the experts’ 
expanded working memory). Experts’ superior ability to encode 
representative information from their domain of expertise and store it in long- 
term memory, such that they can efficiently retrieve meaningful relations, 
provides an alternative to the original account of superior memory in terms of 
larger chunks stored in STM (see Ericsson, Chapter 36, this volume, for a 


discussion of experts’ superior working memory). 


Experts’ Usability of Their Knowledge 


Being able to recall knowledge when explicitly asked does not necessarily 


mean that the individual will always be able to retrieve that knowledge when 


it is relevant. Pioneering investigators (e.g. Feltovich et al., 1984; Jeffries, 


Turner, Polson, & Atwood, 1981) have suggested that a major limitation of 


novices is their inability to access knowledge in relevant situations, even 
when they can retrieve the same knowledge when explicitly cued by the 
experimenter. Problems in knowledge usability may be associated with 
overload or inefficiency in using working (or short-term) memory. 

An alternative proposal about usability of knowledge was subsequently 
put forward by Ericsson and Kintsch (1995). They postulated that experts 
acquire memory skills that are designed to encode relevant information in 
long-term memory (LTM) in a manner that allows automatic retrieval from 
LTM when later needed, as indicated by subsequent activation of certain 
combinations of cues in attention. They argued that experts acquire LTWM 
memory skills that enable them, when they encounter new information (such 
as a new symptom during an interview with a patient), to encode the relevant 
associations such that when yet other related information is encountered 
(such as subsequent information reported by the patient), the expert will 
automatically access relevant aspects of the earlier information to guide 
encoding and reasoning. The key constraint for skilled encoding in LTM is 
that the expert be able to anticipate potential future contexts where the 
encountered information might become relevant. Only then will the expert be 
able to encode encountered information in LTWM in such a way that its 


future relevance is anticipated and the relevant pieces of information can be 


automatically activated when the subsequent relevant contexts are 
encountered. In this model of the experts’ working memory storage in LTM, 
the large capacity of LTM allows the expert to preserve access to a large 
body of relevant information without any need to actively maintain the 
information in a limited general capacity STM (Ericsson, Chapter 36, this 
volume; Gobet & Charness, Chapter 31, this volume; Noice & Noice, 2006; 
Wilding & Valentine, 2006). 


Functionality of Expert Representations Extends to Entire 
Activities, Processes 
The functional nature of experts’ task representations that we mentioned 
earlier extends to entire activities or events. Ericsson and Kintsch (1995) 
proposed that experts acquire skills for encoding new relevant information in 
LTWM to allow direct access when it is relevant and to support the continual 
updating of a mental model of the current situation — akin to the situational 
models created by readers when they read books (see Endsley, Chapter 37, 
this volume, on the expert’s superior ability to monitor the current situation — 
“situational awareness”). This general theoretical framework can account for 
the slow acquisition of abstract representations that support planning, 


reasoning, monitoring, and evaluation (Ericsson, Patel, & Kintsch, 2000). For 


example, studies of expert firefighters have shown that experts interpret any 
scene of a fire dynamically, in terms of what likely preceded it and how it 
will likely evolve. This kind of understanding supports efforts to intervene in 
the fire. Novices interpret these scenes in terms of perceptually salient 


characteristics, for example, color and intensity (Klein, 1998; see also Mosier 


et al., Chapter 25, this volume). In similar manner, expert physicians 
represent diseases as an extended process involving enabling conditions 
(conditions that incline a patient toward a disease), faults (the actual 
pathophysiological or abnormal anatomical features involved in the disease), 
and consequences (the observable signs and symptoms spawned by the 
fault/s) — a so-called “Illness Script” (Feltovich & Barrows, 1984; Charlin, 


Boshuizen, Custers, & Feltovich, 2007). Studies of expert surgeons have 


shown that some actions within a surgery have no value for immediate 


purposes, but are made in order to make some later move more efficient or 


effective (Koschmann, LeBaron, Goodwin, & Feltovich, 2001). The research 


on expert chess players shows consistent evidence for extensive planning and 
evaluation of consequences of alternative move sequences (see Ericsson, 
Chapter 36, this volume, and Gobet & Charness, Chapter 31, this volume). 
Furthermore, there is considerable evidence pertaining to experts’ elaborated 
encoding of the current situation, such as in situational awareness (Endsley, 
Chapter 37, this volume), mental models (Durso, Dattel, & Pop, Chapter 20, 
this volume), and LTWM (Noice & Noice, 2006). 


Analyzing Superior Performance 


Once one has accepted that experts can acquire, build, and modify the 
structure of their cognitive processes, it becomes a question of how their 
superior performance is mediated and how the associated mechanisms are 
either acquired or reflect innate differences. In the expert performance 
approach the first step, as we have noted, involves identifying representative 
tasks and reproducing the experts’ consistent superior performance in a 
controlled laboratory situation (see Ericsson, Chapters 12, 36, and 38, this 
volume). A good example is to have athletes or other individuals generate 
actions in a simulator or respond immediately to a video sequence. In the 
second step, investigators record observable information on the experts’ and 
less skilled participants’ processes by collecting think-aloud protocols and 
patterns of eye fixations. For example, an elite tennis player can anticipate 
(better than chance) where a tennis ball will land before the server has made 
ball contact; this implies an ability to anticipate based on cues in the server’s 
preparatory movements. Such studies and analyses have shown that the 
structure of knowledge is but one of many different types of mental 
representations and skills, and different types of physiological adaptations, 


that account for the observed expert performance. 


Reflection and Mental Representations Mediate Expertise during 
Execution, Evaluation, and Learning 
Another challenge to the traditional information processing view, with its 
severe constraints on cognitive capacity, concerns the experts’ ability not just 


to perform effectively but also to be able to reflect on their thought processes 


and methods (Glaser & Chi, 1988). The traditional account of reflection 
within the information processing model is that abstract descriptions of plans 
and procedures enable an individual to operate on or manipulate problem- 
solving operations, for example, to modify and adjust them to the current 
situation and context. In addition to abstraction in control and planning, there 
must also be mechanisms for maintaining information to allow efficient back- 
tracking or starting over when lines of reasoning need to be modified or 
abandoned. 

The traditional view, especially given severe STM constraints, has 
difficulties in accounting for the possibility that experts might be disrupted or 
otherwise forced to restart their planning. More recent research has shown 
that experts are far more able to maintain large amounts of information in 
working memory. For example, chess masters are able to play chess games 
with a quality that approaches that of normal chess playing under blindfolded 
conditions in which perceptual access to chess positions is withheld. (For a 


review see Ericsson et al., 2000; Ericsson & Kintsch, 2000.) Chess masters 


are able to follow multiple games when they are presented move by move 
and can recall the locations of all pieces with high levels of accuracy. Chess 
masters are also able to recall a series of different chess positions when they 


are briefly presented (5 seconds per position). 


Reasoning and Self-Monitoring in Expertise 


In studies of expert physicians (e.g. Feltovich, Spiro, & Coulson, 1997), we 


have found that when experts do not know the correct diagnosis for a patient, 
they often can give a plausible description of the underlying pathophysiology 
of a disease; that is, they are able to reason at levels which are more 
fundamental and defensible in terms of the symptoms presented. When 
medical students fail to reach a diagnosis for a patient, their rationale for 
possible alternatives is generally incompatible with the symptoms presented. 
Experts fail gracefully; but when medical students fail their mistakes can be 


major. Vimla Patel and her colleagues (Groen & Patel, 1988; Patel & Groen, 


1991) have found that medical experts are able to explain their diagnoses by 
showing how the presented symptoms are all explained by the proposed 
integrated disease state, whereas less advanced medical students have a more 
piecemeal representation that is less well integrated. 

Research in education has consistently demonstrated the value of self- 
monitoring and regulation in learning (for reviews, see Hacker, Dunlosky, & 
Graesser, 2009; Winne & Nesbit, 2009; Zimmerman, 2008). It is also 


important for experts to test their own understanding and evaluate partial 


solutions to a problem. This kind of planning prevents blind alleys, errors, 
and the need for extensive backup and retraction, thus ensuring overall 
progress to a goal. In addition, these same kinds of self-monitoring and self- 
regulation behaviors are critical throughout the process of acquiring 
knowledge and skills on which expertise depends (MacIntyre, Igour, 


Campbell, Moran, & Matthews, 2014). For example, one study of elite 


endurance runners revealed a detailed specification of their self-regulatory 


planning, monitoring, and adjustment processes, and how they contributed to 
their success (Brick, MacIntyre, & Campbell, 2015). Another study 
demonstrated that inducing self-monitoring of learning during acquisition of 
surgical skills led to higher performance levels (Gardner, Jabbour, Williams, 
& Huerta, 2015). 

Thus, the mental representations developed by aspiring experts have 
multiple functions. They need to allow efficient and rapid reactions to critical 
situations, and they need to allow modifiability. This includes, for example, 
mechanisms by which a skilled performer adjusts his/her performance to 
changed weather conditions, such as a tennis player dealing with rain and/or 
wind, or adjusts to unique characteristics of the place of performance, such as 
musicians adjusting their performance to the acoustics of the music hall. 
Furthermore, these representations need to be amenable to change so aspiring 
expert performers can improve aspects and gradually refine their skills and 


their monitoring representations. 


Routine versus Controlled Processing in Expertise 


Experts, for the most part, work in the realm of the familiar (for them, not for 
people in general) and are often able to generate adequate actions by rapid 
recognition-based problem-solving, perhaps followed by more effortful 
reasoning (Chi et al., 1981; Hatano & Inagaki, 1986; Klein, Calderwood, & 


Clinton-Cirocco, 1986; Simon, 1990). In their early work with chess experts, 


Chase and Simon (1973a) conjectured that “the most important processes 


underlying chess mastery are these visual-perceptual processes rather than the 
subsequent logical-deductive thinking processes” (p. 215). With respect to 
expertise, the recognition-based components of skill are sometimes referred 
to as intuitive or routinized while the subsequent forms are called controlled, 
though the operational definition of these terms varies considerably, 
depending on the particular level of explanation and mechanisms under 
discussion. We are referring to the general behavioral forms and temporal 
scales in reasoning, though even the concept of “fast” is necessarily fluid and 
task- or strategy-contingent (Wolfe, 1998). This is in accordance with the 
general assumption that extended experience and practice lead to task- 
specific adaptations over time, wherein conscious deliberation and attention 
is reduced as more commonly encountered and interrelated cues and concepts 
are “automated” toward a more recognition-based behavior (Ericsson et al., 
1993). Biological insights are emerging into how these adaptations- 


automatizations occur (Ashby & Crossley, 2012; Hélie & Cousineau, 2011). 


For example, it has been demonstrated that the speed of such recognition can 
be remarkably fast for trained tasks (Liu, Agam, Madsen, & Kreiman, 2009). 


Such perceptual “expertise effects” have also demonstrated that normally 


face-selective voxels in the Fusiform Face Area of the brain can be highly 
responsive to non-face, but perceptually relevant objects sensitive to 
experience, beyond what can be explained by simple attentional shifts, such 
as expertise in car recognition (McGugin, Newton, Gore, & Gauthier, 2014). 
However, if highly experienced individuals should encounter more 
complex or rare situations (“rare” or “complex” for them, making such an 
encounter progressively unlikely, as more and more cases get brought into 
the realm of the familiar), this type of rapid recognition-based responding 
may lead to incorrect or inferior actions. Individuals who are concerned with 
attaining and maintaining highly accurate performance will therefore 
counteract this tendency to automation by developing more refined mental 
representations that detect minor, but consequential, deviations from the more 
familiar cases (Ericsson, Chapter 38, this volume; Feltovich et al., 1997). 
Several investigators have studied the interplay of intuitive judgments 
and deliberation. Kruglanski and Gigerenzer (2011) argue that both intuitive 
and deliberate judgments operate in a common rule-based architecture, where 
ecological fit of the selected rules determines the behavioral outcomes, 
including suboptimal behavior of intuition. Researchers have found that even 
skilled chess players tend to be inflexible and select recognized opportunities 
to make a good move, whereas highly skilled chess players discover the more 
superior moves for the same chess positions (Bilali¢é, McLeod, & Gobet, 
2008). This suggests that expertise need not lead to inflexibility, but that 
flexibility might be retained at only the highest levels of expertise (see also 
Feltovich et al., 1997). Similarly, a more recent study of chess players by 
Moxley, Ericsson, Charness, and Krampe (2012) had chess players think 
aloud while selecting the best moves for chess positions differing in 


complexity. For easy problems the most skilled chess players selected 


superior moves as the first verbalized moves and the same moves were 
typically selected even after additional time of thinking and searching or 
alternative more superior moves. For more complex problems, the experts’ 
initial verbalized move was later changed by the discovery of a better 
alternative move after subsequent thinking and planning (see also Feltovich 
et al., 1984, for experts’ flexible performance in “garden-path” problem- 
solving designed to trap the problem-solvers). The less skilled players 
showed that their move selection improved for the easy problems with 
additional time. 

These findings are consistent with the finding that experts’ move 
selection under speeded conditions, such as blitz chess (about 5 seconds per 
move), is not as good, on the average, as under normal conditions when they 
might have as much as a minute or more to plan for the best possible move 
(see Ericsson, Chapter 36, this volume; Gobet & Charness, Chapter 31, this 


volume). 


Experience Alone is Not Sufficient for the Development of Expertise 
and Expert Performance 
Most everyday skills are relatively easy to acquire, at least to an acceptable 
level. Adults often learn to drive a car, type, play chess, ski, and play (bad) 
golf to a basic level within weeks or months. It is usually possible to explain 
what an individual needs to know about a given skill, such as rules and 
procedures, within a few hours. Once individuals have learned the underlying 
structure of the activity and what aspects they must attend to, they often focus 
on attaining a functional level of performance. This is often attained within 
less than 50 hours of practice. At this point, an acceptable standard of 
performance can be generated without much need for more effortful 
attention, and execution of the everyday activity has attained many 
characteristics of the automated performance we discussed previously 


(Anderson, 1982, 1987; Fitts & Posner, 1967), and requires only minimal 


effort. 

Once this acceptable performance has been attained, gaining further 
routine experience does not seem to improve performance. For example, 
some individuals play tennis or golf several times a week for decades without 
getting much better. Similarly, many professionals accumulate years or 
decades of experience without increasing their objective performance as 
teachers, nurses, or psychotherapists (see Ericsson, Chapter 38, this volume). 
Reviews (Ericsson & Lehmann, 1996; Ericsson, Whyte, & Ward, 2007) have 


shown that the length of domain experience is often unrelated to 


improvements in professional performance and in some cases the time since 


graduation is even associated with decrements in performance, most likely 


due to forgetting (Choudhry, Fletcher, & Soumerai, 2005; Ericsson, 2015, 


and Chapter 38, this volume). 

In direct contrast to reaching rapidly an asymptote for performance, 
would-be and expert performers keep improving their performance for years 
and decades. Research shows that these experts engage in purposeful 
practice, for which they design particular training environments where they 
practice to attain specific changes. In those cases where this practice is 
supervised by a teacher, Ericsson et al. (1993) referred to it as deliberate 
practice. In order to attain complex representations and skills it is necessary 
to engage in specialized training activities to build and refine these 
mechanisms that mediate the observed superior performance (Ericsson, 
Chapter 38, this volume). 

Research shows that to improve performance it is necessary to seek out 
practice activities that allow individuals to work on improving specific 
aspects, with the help of a teacher, in a protected environment with 
opportunities for reflection, exploration of alternatives, and problem-solving 
—as well as repetition with informative feedback. 

In this handbook, and its previous edition, several chapters discuss the 
effectiveness of this type of deliberate practice in attaining elite and expert 
levels of performance (Ericsson, Chapter 38, this volume; see also 
Zimmerman, 2008), in software design (Sonnentag, Niessen, & Volmer, 
2006), in training with simulators (Ward et al., 2006), in maintaining 
performance in older experts (Krampe & Charness, Chapter 42, this volume), 
and in creative activities (Weisberg, Chapter 41, this volume). Other chapters 
in this volume review evidence on the relationship between deliberate 
practice and the development of expertise in particular domains, such as 


professional writing (Kellogg, Chapter 23), music performance (Lehmann et 


al., Chapter 28), sports (Williams et al., Chapter 34), chess (Gobet & 
Charness, Chapter 31), and mathematical calculation (Butterworth, Chapter 
32). 


Summary and Concluding Remarks 


The theoretical interest in expertise and expert performance is based on the 
assumption that there are shared psychological constraints and adaptations in 
the structure and acquisition of expert performance across different domains. 
The theory of Simon and Chase (1973) proposed that the invariant limits on 
information processing and STM severely constrained how expert skill is 
acquired. They proposed a theory based on the accumulation through 
experience of increasingly complex chunks and pattern—action associations. 
This theory emphasized the acquired nature of expertise and focused on the 
long time required to reach elite levels and the learning processes sufficient to 
gradually accumulate the large body of prerequisite patterns and knowledge. 
This view of expertise offered the hope that it would be possible to extract 
the accumulated knowledge and rules of experts and then use this knowledge 
to more efficiently train future experts and, thus, reduce the decade or more 
of experience and training required for elite performance (Buchanan et al., 
Chapter 7, this volume). Efforts were even made to encode the extracted 
knowledge in computer models and to build expert systems that could 
duplicate the performance of the experts. 

Subsequent research on extended training revealed that it is possible to 
acquire skills that effectively alter or, at least, circumvent the processing 
limits of attention and working memory. Studies of expertise focused initially 
on the expert’s representation and memory for knowledge. As research 
started to examine and model experts’ superior performance on representative 


tasks, it became clear that their mediating complex representations and 


mechanisms could not be acquired by mere experience — living in a cave does 
not make one a geologist (Ericsson, Prietula, & Cokely, 2007). Research to 
determine how individuals achieve expert performance, rather than mere 
mediocre achievement, revealed that expert and elite performers seek out 
teachers and engage in specially designed training activities (deliberate 
practice). The future expert performers need to acquire representations and 
mechanisms that allow them to monitor, control, and evaluate their own 
performance, so they can gradually modify their own mechanisms while 
engaging in training tasks that provide feedback on performance, as well as 
opportunities for repetition and gradual refinement. 

The discovery of the complex structure of the mechanisms that execute 
expert performance and mediate its continued improvement has had positive 
and negative implications. On the negative side, it has pretty much dispelled 
the hope that expert performance can easily be captured and that the decade- 
long training to become an expert can be dramatically reduced (but see 


Hoffman et al., 2014, for examples that do demonstrate acceleration and the 


conditions for acceleration, on the developmental path toward expertise). 
With regard to acceleration, all the paths to expert performance appear to 
require substantial extended effortful practice, but can benefit from 
technologies and training programs that compact or accelerate effortful 
representative experience (e.g. with representative cases) and that provide 
appropriate feedback and guidance. Effortless mastery of expertise, for 
example just teaching learners “expert ways,” is just a myth. This myth 
cannot explain the gradual acquisition, through adaptation, of the 
mechanisms and adaptations that mediate skilled and expert performance. 
Even more importantly, the insufficiency of the traditional school system is 


becoming apparent. It is not reasonable to teach students knowledge and rules 


about a domain, such as programming, medicine, and economics, and then 
expect them to be able to convert their knowledge into effective professional 
skills by additional mere experience in the pertinent domain. Schools need to 
help students acquire the skills and mechanisms for proficient performance in 
the domain under the supervision of teachers. They also need help to acquire 
mental representations that allow them to monitor and correct their 
performance, and that will allow them gradually to take over control of the 
learning of their professional skills and enable them to design deliberate 
practice activities that produce continued improvement. 

On the positive side, the discovery of effective training methods for 
acquiring complex cognitive mechanisms has allowed investigators to 
propose types of training that appear to allow individuals to acquire levels of 
performance that were previously thought to be unobtainable, except for the 
elite group of the innately talented. The study of the development of expert 
performers provides evidence on how they modified or circumvented 
different types of psychological and physiological constraints. It should be 
possible for one type of expert in one domain, such as surgery, to learn from 
how other experts in music or sports, for instance, have designed successful 
training procedures for mastering various aspects of perceptual-motor 
procedures, and to learn the amount of practice needed to reach specified 
levels of mastery. If someone is interested, for instance, in whether a certain 
type of perceptual discrimination can ever be made reliably, and how much 
and what type of training would be required to achieve this, then one should 
in the future be able to turn to a body of knowledge of documented expert 
performance. Our vision is that the study of expert performance will become 
a science of learning and the human adaptations that are possible in response 


to specialized extended training. At the same time, as our understanding of 


the constraints on acquiring high levels of performance in any domain 
becomes clearer, and the similarities of those constraints across many 
different domains are identified, the study of the acquisition of expert 
performance will offer a microcosm for how various types of training can 
improve human performance and provide insights into the potential for 
human achievement. 

The study of expert performance is not merely concerned with the 
ultimate limits of performance, but also with earlier stages of development 
through which every future performer needs to pass. There is now research 
emerging on how future expert performers will acquire initial and 
intermediate levels of performance. Attaining these intermediate levels may 
be an appropriate goal for people in general and for systems of general 
education (e.g. recreational athletes, patrons of the arts). However, knowing 
how to achieve certain goals is no guarantee that people will be successful, as 
we know from studies of dieting and exercise. On the other hand, when the 
goal is truly elite achievement, the study of expert performance offers a 
unique source of data that is likely to help us understand the necessary factors 
for success, including the social and motivational factors that push and pull 


people to engage in the requisite daunting regimes of training. 
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AI and Expert Systems: Foundational Ideas 


Artificial Intelligence (AI) sprang from the startling idea that computers 
could be intelligent and has pursued as its fundamental scientific goal 
understanding the mechanisms of intelligent behavior, whether in machines 
or people. 

One branch of that exploration — variously called expert systems, or 
knowledge-based systems — has focused on constructing programs intended 
to embody a robust model of human expertise in a domain, using a particular 
set of architectural choices and construction techniques. The first element of 
this — the goal of a model of expertise — means these systems are focused on 
getting the answer right. Other research seeks to model cognition more 


broadly (e.g. Newell & Simon, 1972), which includes in its models aspects of 


human cognition that produce errors in performance, such as limited short- 
term memory, forgetting, and biases. Models that produce the same sort of 
mistakes that people do can provide useful insight into human thinking, but 
the focus in knowledge-based systems is on accuracy. 

It is likewise important that these programs are intended to be models of 
human expertise, i.e. they are designed to use the same sort of concepts and 
reasoning techniques that people do. This is in some ways limiting: even a 
brief glance at the long list of techniques used in scheduling, optimization, 
planning, and other fields reveals how many powerful (and not necessarily 
intuitive) problem-solving and inference methods there are. But the focus on 
human expertise makes it enormously more easy to create systems that are 


transparent, i.e. able to explain their reasoning. If the system’s concepts and 


reasoning techniques are familiar to people, transparency can arise in part 
from having the system recount its actions. Also, transparency results in 
easier Maintenance and improvement over time. 

One of the key architectural properties of these systems is the separation 
of their two key components: the knowledge base and reasoning or inference 
engine (Buchanan & Smith, 1988). The inference engine of an expert system 
is the machinery that applies that knowledge to the task at hand. The 
knowledge base of an expert system contains both factual and heuristic 


knowledge. 


e Factual knowledge is that knowledge of the task and the domain that 
is widely shared, typically found in textbooks or journals, and 
commonly agreed upon by those knowledgeable in the particular 
field. 


e Heuristic knowledge is the less rigorous, more experiential, more 
judgmental knowledge of performance. In contrast to factual 
knowledge, heuristic knowledge is rarely discussed, and is largely 
individualistic. It is the knowledge of good practice, good judgment, 
and plausible reasoning in the domain. It is the knowledge that 


underlies the art of good guessing (Polya, 1954). Although Polanyi 


(1958) and others have asserted that much expertise relies on tacit 
knowledge that cannot be articulated, the knowledge engineering 


view of expert systems is that tacit knowledge is explicable. 


Keeping these two factors distinct can make it far easier to augment 
what the system knows, and emphasizes capturing domain knowledge in 
terms of what to know, distinct from how to employ it. The knowledge base 


also typically contains explicit representations of knowledge (Davis, Shrobe, 


& Szolovits, 1993), i.e. knowledge captured in a form that makes it easier to 
examine and modify. One variety of representation widely used in the early 
systems was simple IF/THEN rules that specified a precondition and an 
action that could justifiably be carried out if the precondition was met. This 
simple form is another element in establishing transparency, and facilitates 
knowledge acquisition. 

The emphasis on extensive amounts of domain-specific knowledge and 
relatively modest inference methods is characteristic of these systems. It 
reflects in part the shift to the view that the wellspring of high levels of 
performance is specialized knowledge, not general inference methods. This 
has been called the knowledge-based paradigm in AI (Goldstein & Papert, 
1977). It is also the origin of the term knowledge based, which reflects the 
belief that the performance of these systems is based primarily on knowledge, 
rather than on the inference or problem-solving methods they use. 

Expert systems are a subclass of knowledge-based systems, notably 
those that seek to deliver expert-level performance. Historically, expert 
systems were the starting place for much of this work, as some of the earliest 
efforts aimed at real-world problems that required substantial amounts of 
knowledge and aimed at achieving truly expert-level performance, e.g. 
determining chemical structures from analytic data (Lindsay, Buchanan, 
Feigenbaum, & Lederberg, 1980) or diagnosing causes and recommending 
therapies for infectious diseases (Buchanan & Shortliffe, 1984). It soon 
became clear that the knowledge-based paradigm held even at less expert 
levels of performance, leading to the notion of knowledge-based systems in 
general. 

These systems are also characterized by their construction technique. As 


models of human expertise, they are often developed by debriefing a subject 


matter expert (“SME”), whose knowledge is then represented in an explicit 
form amenable to being examined and modified as the system is tuned. The 
routine use of human experts as the source of knowledge for the systems was 
another factor contributing to calling them expert systems. In the language of 


Kahneman’s Thinking, fast and slow (Kahneman, 2011), the SME articulates 


knowledge in the explicit form that is taught to beginners — the way a careful 
student should proceed — not the abbreviated form that allows fast thinking 
which may include shortcuts not sanctioned by the textbook, “slow” version. 

While many AI programs use substantial amounts of knowledge, e.g. 
about chess or mathematics, expert systems rely on symbolic knowledge 
acquired from sources other than a programmer’s own knowledge or 
introspection, thus forcing a level of perspicuity beyond clever algorithmic 
encoding. 

In the language of knowledge-based systems, task refers to some goal- 
oriented, problem-solving activity; domain refers to the subject area within 
which the task is being performed. Typical tasks include diagnosis, planning, 
scheduling, configuration, design, and advice-giving. The application of task- 
specific methods within a domain of expertise, then, will blend general 
knowledge about how to solve a particular type of problem, e.g. diagnosis or 
planning, with specific knowledge about the instantiation of the problem in a 
domain, e.g. troubleshooting (or diagnosing) failures in automobile engines 
or scheduling (planning) aircraft crew assignments. 

As with human experts whose expertise is based on cognitive skills, 
rather than perceptual or motor skills, a knowledge-based system is expected 
to exhibit several important properties (Berg & Sternberg, 1992; Shanteau, 
1988): 


e Problem-solving at high levels of ability, well above the performance 
levels of competent practitioners and novices, even in the face of 


incomplete or incorrect descriptions of problems. 


e An architecture that separates facts about the subject matter domain 


from procedures and strategies that use those facts. 


e A capacity to explain the relevant factors in solving a problem and to 


explain items in its knowledge base. 


e A capacity to modify its knowledge base and to integrate new 


knowledge into the knowledge base. 


One of the most important contributions of knowledge-based systems to the 
study of expertise has been to provide tools for building testable models and 
thus determining characteristics of expert problem-solvers (e.g. Elstein, 
Shulman, & Sprafka, 1978; Larkin, McDermott, Simon, & Simon, 1980; 
Pauker & Szolovits, 1977). With a computer program that can be run 
repeatedly under varying conditions, questions and claims about the 
importance of different parts of the model can be examined without trying to 
manipulate human subjects (Buchanan, 1994). For example, are there 
measurable differences in performance when goal-driven and data-driven 
strategies are used for problem-solving? 

More generally, computer programs provide not just proofs of concept 
but a means to experiment with variations on the mechanisms that would be 
impossible, or unethical, with human subjects (Buchanan, 1994). 

Knowledge-based systems have also brought new methods and new 
questions into the study of expertise, and into the science and engineering of 


Al. For instance, is performance alone sufficient to call a system, or a person, 


an expert? The DeepBlue chess program (DeepBlue, 2005) is a case in point. 


Although it won a celebrated match against the reigning world champion, its 
success was probably due more to the number of possibilities it could 
consider at each move than to its knowledge of chess strategy and tactics. 
How much does the speed of performance matter in the definition of 


expertise (Anderson, 1982; Arocha & Patel, 1995)? How does one measure 


the amount of knowledge used by a person or a program? 


A Brief History of AI and Knowledge-Based 
Systems 


Knowledge-based systems are based on the computational techniques of AI. 
From its beginnings as a working science in 1956, AI has been a growing 
collection of ideas about how to build computers that exhibit intelligence. As 
mentioned, one major branch of AI, the psychology branch, sought to 
understand and faithfully simulate the problem-solving methods of humans. 
A second major branch of AI, the engineering branch, sought to invent 
methods that computers could use for intelligent problem-solving, whether or 


not used by humans (Feigenbaum & Feldman, 1963). In both branches, an 


important source of data and inspiration was the human problem-solver, and 
both have contributed to the study of expert systems. 

In the earliest phase of AI, roughly 1950-65, there was much emphasis 
on defining efficient symbol manipulation techniques, finding efficient means 
to search a problem space, and defining general-purpose heuristics for 
pruning and evaluating branches of a search tree. The early programs were 
demonstrations of these core ideas in problem areas that were acknowledged 
to require intelligence. For example, in 1956-57, Newell, Shaw, and Simon’s 
Logic Theory Program found two novel and interesting proofs to theorems in 
Whitehead and Russell’s Principia Mathematica; in 1957-58, Gelernter’s 
Geometry Theorem Proving Program showed superb performance in the New 
York State Regents Examination in Plane Geometry; and by 1963 Samuel’s 
Checker Playing Program had beaten one of the best checker players in the 


USA (Samuel, 1959). Samuel’s work is especially interesting, given the 


expert systems work that was to come, because he chose the components of 
the feature vector used to evaluate the goodness of a board position by 
extensively interviewing master checker players. 

Theorem proving within formal logic was also a major focus in AI in the 
1960s. It appeared to be a universal method for solving problems in any task 
domain and was especially attractive after Robinson invented an efficient 
method for proof-by-contradiction, called the Resolution Method (Robinson, 
1968). To some, it seemed that the main problem of creating intelligent 


computers had been solved (Nilsson, 1995), because in all of this early work, 


intelligence was considered to be due more to the methods than to the 
knowledge, including the methods of search, means—end analysis, 
backtracking, and analogical reasoning. 

Others in AI experimented with knowledge-rich programs in a quest for 
powerful behavior. With knowledge-rich programs, expertise is seen to lie in 
the domain-specific and common-sense facts, assumptions, and heuristics in a 
program’s knowledge base, implementing a computational form of Francis 
Bacon’s assertion (1597) that in knowledge lies power. 

A program’s knowledge representation formalizes and organizes the 
program’s knowledge. One widely used representation in expert systems, is 
the conditional rule, “IF A , THEN B,” sometimes knows as a production rule, 
or simply rule. Expert systems whose knowledge is represented in rule form 
are called rule-based systems (Buchanan and Shortliffe, 1984). The 
antecedent lists a set of conditions in some logical combination. The 
consequent may name a complex action, and when the conditions are 
satisfied, the consequent can be concluded, or its problem-solving action 


taken. A surprising result of this work has been that reasoning methods in 


programs achieving high levels of expertise can be quite simple, often little 


more than modus ponens (i.e. if A, and A implies B, then B). 


The Emergence of the Expert Systems Focus 
in AI Research, 1965-1975 


Beginning in the mid-1960s, in particular with the DENDRAL research 
project (Lindsay et al., 1980) at Stanford University, AI research began to 


shift to exploring the power of knowledge in problem-solving. Questions for 
Al arose that were framed in terms of the knowledge required for expert-level 


performance. For example: 


e How could the methods-based approach of earlier AI work be 
augmented by domain-specific knowledge to model human expertise 
in difficult tasks of hypothesis induction? The specific task for 
DENDRAL was to hypothesize organic chemical structures from 
spectral data. Because the task was performed by chemists with 
doctoral degrees, it required expertise that had to be specified and 
represented. And because the experts’ knowledge had to be translated 
into computer-readable representations, the programmers not only had 
to elicit the knowledge from experts, they had to interpret it and 
represent it in ways that would allow easy examination and change. 
Representing it directly in computer code would render it opaque and 
subject to misinterpretation when translated, so it was written in 
straightforward conditional rules, with meaningful predicate names in 
the antecedents and action names in the consequents. The complexity 
of the methods for computing the antecedents and consequents could 
thus be made invisible to the experts looking at and modifying the 


conditional rules. 


e For programs that achieved expert levels of performance, what was 
the source of their power? Relatively speaking, was the power in 
knowledge used, or in the reasoning method used? DENDRAL’s 
methods were essentially heuristic search and modus ponens. At its 
core was Lederberg’s notational algorithm that defined a complete 
and non-redundant search space of chemical structures. But the 
astronomical size of the search space requires powerful heuristics that 
focus the search by linking elements of specific problems to elements 
of the search space. Experts’ knowledge of chemical analysis was 
necessary to infer the search heuristics needed to constrain each 


problem with information from the available data. 


e How could the domain-specific knowledge be represented in a way 
that was modular, easily understandable to both expert-builders and 
end-users, efficient at the engineering stage of knowledge acquisition 
and efficient at run time when reasoning programs were using the 
knowledge ? Production systems — collections of conditional sentences 
with an interpreter no more complex than modus ponens — were in use 
to build psychological simulations of people solving problems of 
various types (Davis & King, 1984). These were suitable for encoding 
DENDRAL’s specialized chemistry knowledge (of mass spectrometry 
in particular) because they were highly modular and allowed use of 
the experts’ vocabulary. Each rule, then, could be understood singly 
and within groups of similar rules both as declarative statements and 


as steps within the interpretive process. 


e Are there any new AI methods, or combinations of old methods, to 


discover in relation to the induction task? Tasks in empirical science 


require methods for assessing the degree of confirmation of 
alternative hypotheses. Most researchers at the time were studying 
problem-solving in the context of games, puzzles, and mathematics, 
where a suggested solution was either correct or not and little 
knowledge of a specific subject area was required. Knowledge-based 
methods were applied to the domains of logic and mathematics, to 
make smart and useful aids to humans in symbolic algebra and 


calculus (e.g. Hearn, 1966; Moses, 1971). These emphasized the 


symbol manipulation aspects of mathematics and were influential in 
the development of the widely used software package Mathematica. 
Or they were looking for statistical methods to assess probabilities. 
Because experts’ knowledge provides power to expert systems, it was 
important to find representations of it that could be examined by the 
experts and modified to deliver high performance. Simple conditional 
rules — augmented with a degree of evidential support — provided both 
the modularity and effectiveness for both understandability and high 
performance. In one of the first real-world experiments, the rule-based 
representation enabled a learning program, Meta-DENDRAL, to 
discover new rules of mass spectrometry that were subsequently 
published in the refereed literature of chemistry (Buchanan et al., 
1976). 


By 1977, the DENDRAL project and its siblings in chemistry, medicine, 
and other areas of expertise (e.g. Michie, 1979; Buchanan & Shortliffe, 1984) 


had demonstrated the effectiveness of the knowledge-based paradigm. The 


science and applications of AI work were dominated by expert systems in the 


period 1974—84. In 1985, the eminent computer scientist, cognitive scientist, 


and co-founder of AI, Allen Newell of Carnegie-Mellon University, wrote: 


There is no doubt, as far as I am concerned, that the development of 
expert systems is the major advance in the field during the last decade. 
The emergence of expert systems has transformed the enterprise of AI, 
not only because it has been the main driver of the current wave of 
commercialization of AI, but because it has set the major scientific 
problems for AI for the next few years — namely, to assimilate expert 


systems into the general body of scientific knowledge in AI. 


(A. Newell quoted in Bobrow & Hayes, 1985, p. 385) 


Claims about Expertise Resulting from Work 
on Expert Systems 


Modeling expertise in working programs has brought several issues into 


focus (Hayes-Roth, Waterman, & Lenat, 1983). As one would expect, the 


two main areas for research on expert systems are also central issues in AI: 
knowledge representation and reasoning. In addition, three other major lines 
of work take on extra importance in dealing with expert systems: knowledge 
acquisition, explanation, and validation. Within each of these areas many 
issues have been explored in both psychology and AI; for some there have 
been substantial results (e.g. Forbus, Usher, Lovett, Lockwood, & Wetzel, 
1997; Chi, Glaser, & Farr, 1988), while for others these issues are still 


driving new research. 


The following 13 claims summarize results from several decades of 


research and applications. 


Knowledge of a Domain is More Essential for Expertise than 
Complex Inferential Procedures for Many Tasks 


From the line of research starting with the classic study by Simon and Chase 


(1973), it is now recognized that expertise is truly domain-specific and does 


not transfer from one domain to another. Chess masters, unlike novices, 
know about 50,000 patterns that they use to recognize chess situations 


(Simon & Chase, 1973), but their problem-solving expertise does not 


generalize to other tasks. Medical doctors who are expert diagnosticians are 
no better or worse at troubleshooting automobile engine problems than the 
rest of us. Expertise depends on well-organized, specialized knowledge much 
more than on either superior memory skill, which would transfer, or general 
problem-solving ability, which also would transfer. 


Work on transfer learning (Pan & Yang, 2010) addresses this dilemma. 


One would expect more success on the side of task-specific knowledge (e.g. 
knowledge of the process of diagnosis) than domain-specific knowledge (e.g. 
medicine). In the expert systems world, MYCIN’s procedure for diagnosis 
was successfully transferred (manually) to other domains: the domain- 
specific knowledge was stripped out of the program, leaving only a “shell” 
system capable of diagnosis and troubleshooting in other domains. This 
program, named “EMYCIN” for “Essential MYCIN,” was the first expert 
system “shell”; CLIPS, another rule-based shell, is one of several widely used 


systems (CLIPS, 2004). The key to this transfer was the separation of the 


domain-specific rules from the procedure that applied them and the strategy 


under which the application should be done. 


Knowledge means things like: terms for entities, descriptions of those 
entities, relationships that organize the terms and entities for reasoning, 
symbolic concepts, abstractions, symbolic models of basic processes, 
fundamental data, a large body of remembered instances, analogies, heuristics 
for good guessing, key sources and other experts, among many other things. 

In contrast, programs that are rich in general inference methods, some of 
which may even have some of the theorem-proving power of mathematical 
logic, but are poor in domain-specific knowledge, can behave expertly on 
almost no tasks. The Prolog computing language is based on theorem proving 
as an inference method and has been used for building many successful 
expert systems. Their power also comes from domain-specific knowledge in 
the declarative statements provided for each problem (Robinson & Sibert, 
1982). 


There is More to Expertise than is Captured in the Oversimplified 

Model of Knowledge Base + Inference Engine 
The simple two-compartment model did highlight the importance of 
separation so that the relative simplicity of the inference engine could be 
demonstrated. And it also highlighted the important issues of representing 
and acquiring expertise in declarative knowledge structures. IF-THEN rules, 
for example, seem “natural” for stating the inferential knowledge needed to 
diagnose the causes of many medical problems or for classifying borrowers 
into levels of credit risk. 

It was clear, however, that other important kinds of knowledge 
structures are used by experts in addition to simple inference rules. In the 
decades after the successful demonstration of MYCIN, and even within that 
program, much work has focused on the use of spatial, temporal, taxonomic, 
and causal relationships among types of objects and events in the domain. 
Diagrams are known to be useful for human problem-solving (Chang & 


Forbus, 2014; Forbus et al., 2011; Polya, 1954), but their use by computers is 


still only partially understood (Hammond & Davis, 2004; Lindsay, 2012). 


Experts also use knowledge of mechanisms to account for causal 
relationships, side effects, and anomalies when observed facts are at odds 
with the accepted wisdom. 

Lacking this kind of richness, knowledge-based systems do not perform 
well in the face of unanticipated contingencies; human experts do much 
better. The first generation of knowledge-based systems exhibited only a little 
flexibility. Within the sphere of “known unknowns” they performed 


admirably; but outside of their restricted scope they were brittle. That is, 


when the details of a specific problem crossed a boundary of a system’s 
knowledge, the system’s behavior went from extremely competent to 
incompetent very quickly (Davis, 1989). To overcome such brittleness and 
deal with unanticipated situations, researchers have focused on reasoning 


from models, principles, and causal mechanisms (Davis, 1984). 


Meta-Level Knowledge about the Strategies, Contextual Cues, and 
Appropriateness for Using Specific Items of Knowledge is an 
Important Part of Expertise 
Strategic knowledge is important because of its power: experts use more 
efficient problem-solving strategies than novices. Not only do we expect 
experts’ answers to be better than those of novices, we expect their chain of 

reasoning to be more focused and more efficient. 

This capability is replicated to some extent in expert systems through 
meta-level knowledge. For example, MYCIN’s diagnostic strategy was 
predominantly backward chaining: starting with the goal of recommending 
therapy for a patient with an infection, MYCIN works backward at what it 
needs to know to do that — recursively until the answers to what it needs to 
know can be found by asking a doctor or nurse. This conveys a sense of 
purpose to the doctor or nurse using the program. However, MYCIN was also 
given meta-knowledge to direct the lines of reasoning even further, e.g. to 
indicate the order in which to pursue different goals. Meta-knowledge in the 
program, as with experts, also told MYCIN whether enough information was 
available on a case to warrant a conclusion or whether it had enough 


knowledge relevant to a case to attempt solving it at all (Davis, 1980). 


Expertise Requires Dealing with Facts about People and Things in 
the World that are Almost Always Incomplete and Uncertain 
Work on knowledge-based systems made it abundantly clear that any model 
of expertise must deal with facts that are uncertain or missing altogether. 
Observational reports, for example, frequently contain errors and partial 
truths. An expert, or an expert system, may fill in reasonable defaults by 
looking at prototypes or by inferring plausible features from others that are 
known. Or it may be possible to ignore the missing information and deal just 
with available data. In real-world problems, the evidence is rarely certain and 
it becomes important to accumulate support from several pieces of data. The 
set of methods for using uncertain knowledge in combination with uncertain 
data in the reasoning process is called reasoning with uncertainty. Knowing 
how to treat incomplete descriptions is a small, but important, part of high 


performance (Moskowitz, Kuipers, & Kassirer, 1988). 


Expertise Also Requires Dealing with the Uncertainty of the 

Knowledge and Assumptions on which Inferences Are Based 
The accepted textbook knowledge and best practices used by experts are 
incomplete in many domains. In medicine, for example, effective treatments 
may be associated with sets of symptoms before their mechanisms are 
known. Moreover, the amount of experiential evidence supporting these 
associations may range from “used for centuries” to “seemed to work in a 
similar case.” In real-world problems, the available problem-solving 
knowledge is often uncertain, and it is important to account for the credibility 
of the associations or the degree to which they can be believed. To deal with 
uncertain inference, a rule may have associated with it a confidence factor or 
a weight indicating how much evidential support the condition provides for 
inferring the conclusion. The set of methods for reasoning with uncertainty, 
then, must take into account both uncertain knowledge and uncertain data in 
the reasoning process. 

One important method for reasoning with uncertainty combines 
probability statements using Bayes’ Theorem to infer the probabilities 
associated with events or outcomes of interest. Tversky and Kahneman 
(1974) have shown that even expert decision-makers fail to combine 
probability statements rationally, according to Bayes’ Theorem and other 
laws of probability. By contrast, a Bayesian program makes no such 
calculation errors. This helps emphasize the point that expert systems are 
models of human expertise as it ought to be applied, not computational 


models of human performance with all of its shortcomings. 


Several other methods have been introduced for assessing the strength of 
evidence and of the conclusions it supports within expert systems (Buchanan 
& Shortliffe, 1984; Gordon & Shortliffe, 1985; Pearl, 2001; Weiss, 
Kulikowski, Amarel, & Safir, 1978; Zadeh, 1965). One of the lessons learned 


from these investigations is that rough estimates of uncertainty often support 


expert-level performance. Moreover, rough estimates avoid the illusion of 
knowing more precise facts than are actually known and serve as reminders 


that the data may support alternative conclusions. 


Eliciting Expert Knowledge is not Necessarily the Same as Eliciting 
Tacit Knowledge 

The process of transferring knowledge to a computer program from an expert 
came to be seen as a much-discussed “bottleneck” because it is difficult and 
time consuming. The first systems were constructed by programmers, who 
came to be known as “knowledge engineers,” interviewing experts and 
transforming what they understood into a machine-usable form. Not 
surprisingly, it became obvious that knowledge engineers with social skills 
were more adept at this than others. However, they also needed to be very 
skilled at thinking through the complexity of finding efficient representations 
and computational procedures to using the knowledge. Eliciting knowledge 
from experts and representing it for use in a knowledge-based system is a 
skill in its own right worthy of investigation (Motta, 2013; Scott, Clayton, & 
Gibson, 1991; Shadbolt & Burton, 1989), but it can be accomplished with 
careful and skilled knowledge engineering (Shaw & Gaines, 1987; Tecuci, 
Marcu, Boicu, & Schum, 2015). 


However, in spite of early reservations about the impossibility of 


eliciting tacit knowledge, it became obvious that for many applications, 
experts did not need to articulate how they actually reasoned, only how they 
believed a careful, rational person or program should reason. 

Iterative refinement of a knowledge base using case presentations has 
been found to be a successful method for eliciting knowledge from an expert 
that might otherwise appear to be inexplicable. Interviewing alone is not as 
successful as interactive discussions of specific problems. However, insofar 


as the knowledge transfer process requires social interaction between 


knowledge engineers and experts (Forsythe & Buchanan, 1992), there will be 
non-technical difficulties that interfere with success. 

Numerous approaches have been taken to ameliorate the difficulties of 
eliciting and translating an expert’s knowledge for use by a computer 
program. Interactive tools have been developed to assist in conceptualizing 
and encoding expertise (Boose, 1989) and to assist in the process of 
knowledge base refinement (Davis, 1979; Pazanni & Brunk, 1991; Tecuci et 
al., 2015; Wilkins, Clancey, & Buchanan, 1987). 

Polanyi’s concept of tacit knowledge (Polanyi, 1958) was thought in the 


1960s to provide a stumbling block, if not an impossibility argument, for 
encoding expertise. We now know this is not the case. Although experts may 
still perform with learned knowledge that has become tacit with use and often 
elided into shortcuts, they can often articulate the knowledge underneath their 
thinking using the kinds of rules and principles that are learned from 
textbooks and that can model their faster-thinking performance. In other 
words, what is often referred to as “intuition” is not a mystery: it is 


knowledge that is unexamined. 


Continued Maintenance of a Knowledge Base is a Key to 
Continuing Success 
Since most interesting tasks requiring expertise are not static, the knowledge 
base requires frequent updating. Organizing a body of knowledge within a 
conceptual framework (called an “ontology”) that is familiar to an expert 
makes it easier to manage and easier to maintain (Chandrasekeran, 


Josephson, & Benjamins, 1999). 


As the knowledge base grows, however, the number of interactions 
among its elements grows with it. Limiting the scope of the problem being 
addressed is key to successful maintenance efforts. In some cases, as with 
medical diagnosis in any sub-specialty area, the relevant knowledge grows in 
spite of a constant scope. Subject area domains with rapidly expanding 
knowledge have proven to be extremely difficult for both expert systems and 
human experts to master and maintain their mastery. 

With expert systems and other computer programs relying on large 
amounts of knowledge, there is some progress in automating the maintenance 
of the knowledge bases. Machine learning has matured to the point that 
knowledge bases for expert systems can sometimes be learned from stored 


descriptions of prior cases (Buchanan & Wilkins, 1993; Pazzani & Brunk, 


1991; Rulequest, 2017). However, performance and understandability are 


improved after an expert reviews and modifies the learned information 
(Ambrosino & Buchanan, 1999; Davis, 1979; Richards & Compton, 1998). 


In any case, the vocabulary and conceptual framework in which the 


experiential data are described are critical to the success of automated 


systems that search for associations in the data, just as they are when experts 
are looking for patterns in data. 

Learning by reading is becoming a possible route for knowledge 
acquisition, now largely limited to Google-style statistical learning from 
massive amounts of text available on the Web. A substantial international 
effort is under way to define, and later distribute, semantic markup languages 
that would empower those who create Web or database entries to give some 
meaning to their text or graphics. The flow of research communications about 
the so-called semantic web (Berners-Lee, Hendler, & Lassila, 2001) are on 
the website www.semanticweb.org. The technology for traversing the Web to 
infer knowledge from the semantic markups is complex, in part because it 
involves semantic structures (ontologies), and needs human assistance, at 
least for the foreseeable future (Hendler & Feigenbaum, 2001). As a result, 


progress has been limited to date. 


Expertise Serves Many Purposes and Can be Categorized Along at 
Least Two Dimensions: Formal vs. Informal Knowledge and Public 
vs. Private 
As described by Forsythe, Osheroff, Buchanan, and Miller (1991), 
knowledge encoded in textbooks and journals is formal and public. It is the 
stuff we usually think of when we think of knowledge acquisition for expert 
systems. But other forms of knowledge, such as heuristics shared among 
members of a lab, tend to be informal and private. This form of expertise can 
be difficult to elicit in part because it is not written down and may not be 
recalled until needed in practice, e.g. who to ask about a specific problem. 
Insofar as expert systems codify knowledge from experts as well as from 
textbooks and journal articles, they are formalizing some informal heuristics 
and moving some privately held knowledge into a more public form, 
available to an enterprise-wide community of practice. This is analogous to 
the “knowledge sharing” that is central to all current knowledge management 
efforts (O’Dell & Hubert, 2011; Smith & Farquhar, 2000). Paradoxically, 


making private knowledge public may destroy its value, as might be the case 


with knowledge about people who are willing to “bend the rules” to expedite 


a process. 


Human Expertise is Not Always Necessary for High Performance 
Problem-Solving 
Although problems involving perceptual or motor skills are difficult to 
rationalize, computers have achieved high, and increasingly more expert, 
levels of performance in some task areas with a variety of computational 
procedures that may have little correspondence with human thinking in these 
tasks. These include both sophisticated mathematics based on detailed 
physical and control models as well as statistical learning from large 
collections of examples. For example, through various mixes of control 
theory, machine learning, perception, and robotics, computers can land a 
jumbo-jet, play ping-pong, and drive a car, truck, or lunar explorer (see, for 


example, Urmson et al., 2009). 


Purely statistical methods work well for perceptual and motor tasks — 
and even for some cognitive tasks like translating among languages — when 
there are very large databases from which to learn (Halevy, Norvig, & 


Pereira, 2009). Building neural networks from massive amounts of data, 


known as “deep learning,” has decoupled human expertise and high 
performance. For example, the 2016 victory of the AlphaGo program over a 
champion GO player demonstrates the power of machine learning (Byford, 


2016), although the learned strategies and tactics are not of use to humans 


learning to play the game. With expert systems, the rationalization for an 
expert’s cognitive performance does not capture the exact path of synapses of 
his or her actual thought, but that level of detail is considered to be 


uninformative for training human novices. 


Expertise in Knowledge-Based Systems, and in Many Humans, is 
Limited to a Circumscribed Frame of Reference 

As with human problem-solving, the knowledge and _ problem-solving 
methods are embedded within a larger conceptual framework that carries 
basic assumptions about the world. To the extent that the context can be 
articulated, knowledge-based systems can be given an ability to determine 
when they are operating at the boundaries of their expertise. Even human 
experts are sometimes blind-sided by making invalid, often implicit, 
assumptions about context. For example, computer and human 
diagnosticians, such as physicians and automobile mechanics, frequently 
assume that a single causal explanation for all the symptoms is preferable to 
an explanation with multiple causes (Occam’s Razor). 

It is apparent that cognition is situated in a much larger world of 
experience than can now be given to knowledge-based systems. There are 
numerous assumptions we all make based on the context of the moment. 
Moreover, an expert’s own experience certainly colors his or her articulation 
of knowledge, with a blurring between the facts that are generally true and 
the facts that are based more on personal experience. A big advantage of 
using computational systems to study expertise is that assumptions such as 
these can be made explicit and changed, while with human experts they may 


remain unexamined for a long time. 


The Specialized Knowledge of Experts Often Rests on a Base of 
Everyday Knowledge 

Although it is possible for people to exhibit extraordinary ability in just one 
area that is largely decoupled from everyday experience, such as chess or 
music, many experts need to interact with people and the world within their 
areas of expertise as well as outside of it. In medicine, for example, experts 
cannot be isolated from what their patients take for granted. Because 
“everyone knows” that a human fetus develops only in human females, it is a 
sign of ignorance for a medical diagnosis program, or a medical student, to 
ask whether a male patient has recently undergone a test for abnormal fetal 
development. 

The task of encoding millions of such facts about the everyday world is 


daunting, but conceivable (Lenat & Feigenbaum, 1987), and at least one large 


project, CYC, is encoding common-sense knowledge. Expert systems have 
been able to finesse the problem of encoding all that knowledge, however, by 
making the assumption that persons using those programs themselves have 
enough common sense to avoid making stupid inferences. To a large extent, 
the circumscribed scope of an application makes this assumption workable in 


practice. 


Expertise is Partly Defined by Experts’ Ability to Explain Their 
Reasoning 
Experts may leap to a conclusion without consciously stepping through a 
chain of inferences. However, we have found that they can, after the fact, 
explain where their conclusions come from, a requirement Plato believed was 
necessary for distinguishing knowledge from mere belief. We expect them to 
be able to teach apprentices how to reason about hard cases and critique their 
own and others’ use of knowledge. We expect knowledge-based systems to 


have some of the same capabilities (Rennels, Shortliffe, & Miller, 1987). 


After all, in order to commit significant resources to a recommended action 
we want to know the justification for it, and our legal system demands that 
decision-makers be able to justify those actions. A person who merely carries 
out the orders of a “black box” cannot lay claim to being an expert — but 
neither can the box. 

Expert systems have demonstrated the ability to show how they reach a 
conclusion by showing the rules that connect inferential steps linking primary 
facts about a case with the program’s conclusions, e.g. its recommendations 
for how to fix a problem, as in the MYCIN program (Buchanan & Shortliffe, 
1984). They can also explain why some facts and inference rules were used 
and not others. And they allow users to query and browse the knowledge base 
in order to see the scope and limits of what the system knows. 

However, expertise in a program rests on implicit assumptions about the 
context in which a program will be used, e.g. what the primary task is, what 
data are available, and what the users of the system can be expected to know 


(Clancey, 1985). In training novices, and in other contexts, it is important to 


be able to convey the assumptions and strategies, and even describe the 
mechanisms on which they rest. Each task and problem-solving context 


dictates the amount of detail that has to be made explicit. 


Expertise is Difficult to Quantify and is Multidimensional 


A widely quoted metric for humans to attain expert-level knowledge and 
skills is the number of hours spent in deliberate practice (Ericsson, Chapter 
36, this volume; Hayes, 1985). We have considered whether there is a 
corresponding measure for the amount of knowledge needed by knowledge- 
based systems for expert-level performance. 

For example, simple counts of the number of rules, predicates, or lines 
of code can measure a program’s size, but we have concluded that the scope 
of the domain and the complexity of the task argue against finding a simple 
measure of a program’s knowledge. In particular, measuring a system’s level 
of expertise by the size of its knowledge base proved to be misleading 
because the grain size of the primitive concepts used can vary widely. For 
example, in MYCIN the concept of degree of sickness could either be a 
primitive, whose value would be filled in by a physician or nurse, or it could 
be inferred from rules using the values of several other primitives such as 
temperature and heart-rate. 

Rather than measuring the size of the knowledge base, MYCIN’s level 
of expertise was measured through a series of evaluations that compared its 


performance to that of humans (Buchanan & Shortliffe, 1984). Because 


expertise in medical diagnosis, as in all other areas, is not precisely defined, 
MYCIN’s performance was ranked by a panel of acknowledged outside 
experts against the performance of several persons, called the practitioners, 
whose probable expertise ranged from novice (a medical student) to 
competent practitioner (physicians without sub-specialty training) to local 


expert (faculty providing the sub-specialty training). The practitioners were 


asked to look at descriptions of randomly selected cases of infections and 
provide therapy recommendations. MYCIN was given information from the 
same descriptions. Then the panel of outside experts was asked whether each 
of the recommendations — from the practitioners and, anonymously, from 
MYCIN -— disagreed with their own recommendation for these cases or was 
therapeutically equivalent. Based on the number of times the outside experts 
said that MYCIN’s recommendation was acceptable, compared with the 
numbers of acceptable recommendations among the practitioners, MYCIN’s 
performance was found to be indistinguishable from that of the local experts, 
and better than the performance of the competent practitioners and the 
novice. 

In this and other task domains for which there is no gold standard of 
correctness, using acknowledged experts to judge the relative expertise of an 
expert system has become a widely used method of evaluation. Numerous 
other methods for judging the appropriateness and correctness of knowledge 


bases have also been proposed (Buchanan, 1995). 


Applications of Knowledge-Based Systems 


In the development of expert systems, each program served as an 
experimental apparatus for testing hypotheses about sufficient mechanisms 
for encoding and using expertise to achieve high levels of performance. 
Experiments with the rule-based paradigm, for example, are described by 
Buchanan and Shortliffe (1984). As the technology matured, its potential 
utility in industry became a question that many leading corporations and 
institutions began to investigate vigorously (Feigenbaum & McCorduck, 
1988). They were interested in the tangible benefits of expert systems, which 
derive from a few basic facts: computers process data and text faster and 
more reliably than people; computational procedures are more consistent than 
people; computer software can be replicated cheaply and easily; expertise is 
scarce and expensive. 


Because of these facts the major benefits become: 


e A speed-up of human professional or semi-professional work — 
typically by a factor of ten and sometimes by a factor of a hundred or 


more. 


e Improved quality of decision making. In some cases, the quality or 
correctness of decisions evaluated after the fact show a tenfold 


improvement. 


e Major internal cost savings. Savings within companies can result from 
quality improvement or more efficient production of goods and 


information, and provide a major motivation for using expert systems. 


e Preservation and dissemination of expertise. Expert systems are used 
to preserve scarce know-how in organizations, to capture the expertise 
of individuals who are retiring, and to preserve corporate know-how 
so that it can be widely distributed to other factories, offices, or plants 


of the company. 
e Wide spectrum of applications. 


Today, knowledge-based systems are widely used as surrogate experts and 
decision making assistants — in business, manufacturing, and service 
industries, in healthcare, education, finance, science, space exploration, and 
defense. A few are highlighted here to illustrate the nature of the task 
domains, selected from the annual conference, Innovative Applications of AI 
(IAAI-89 through IAAI-16). It is important to note that the domains of 
successful applications are generally small and bounded in comparison with 
everything an expert knows. 

Applications of expert systems find their way into most domains of 


human activity involving knowledge work (Engelmore, 1993), and tend to 


cluster into three major classes of tasks: analysis, synthesis, and decision 
making and advice-giving (with many notable outliers). Several exemplars 


are summarized below; others are cited in the references. 


Analysis Tasks: Diagnosis and Troubleshooting of Devices and 
Systems 

This class comprises systems that diagnose faults and suggest corrective 
actions for a malfunctioning device or process. The diagnostic problem can 
be stated in the abstract as: given some evidence of a malfunction, what is the 
underlying problem/reason/cause? 

Medical diagnosis was one of the first knowledge areas to which expert 
systems technology was applied (Pauker & Szolovits, 1977; Pople, Myers, & 
Miller, 1975; Shortliffe, 1976; Weiss et al., 1978), but diagnosis of 


engineered systems quickly surpassed medical diagnosis. A clear example of 


this class of systems is the TED system developed for the army’s M1 Abrams 
tank turbine engine (Hoffman, Baur, Dumer, Hanratty, & Ingham, 1999). 


There are probably more diagnostic applications of expert systems than any 
other type. Other diagnostic systems include telephone help desks and 


equipment troubleshooting. 


Process Monitoring and Control 


Systems in this class analyze real-time data from physical devices with the 
goal of noticing anomalies, predicting trends, and controlling for both 
optimality and failure correction. One of the longest-running commercial 
expert systems (since the 1980s) monitors the operations of 1,200 gas 
turbines, steam turbines, and generators (Thompson et al., 2015). Other 
examples of real-time systems that actively monitor processes can be found 
in steel making, oil refining, and even the control of space probes for space 
exploration (Muratore, Heindel, Murphy, Rasmussen, & McFarland, 1989; 
Nayak & Williams, 1998). 


Synthesis Tasks: Planning and Scheduling 


Systems that fall into this class analyze a set of one or more potentially 
complex and interacting goals in order to determine a set of actions to 
achieve those goals, and/or provide a detailed temporal ordering of those 
actions, taking into account personnel, materials, and other constraints. This 
class has great commercial impact. Maintenance of rapid transit subway, 
airport express, and commuter rail lines in Hong Kong, for example, is 
scheduled by an expert system to avoid disruptions and delays as much as 
possible — a “mission critical” application (Chun & Suen, 2014). A system 


developed for Ford Motor Company manages the automobile manufacturing 


process planning at Ford’s vehicle assembly plants (Rychtyckyj, 1999). Other 
valuable examples involve airline scheduling of flights, personnel, and gates; 


manufacturing job-shop scheduling; and manufacturing process planning. 


Configuration of Manufactured Objects from Subassemblies 


Configuration is historically one of the most important of expert system 
applications, and involves synthesizing a solution to a problem from a set of 
elements related by a set of constraints. Configuration applications were 
pioneered by computer companies as a means of facilitating the manufacture 


of semi-custom minicomputers (McDermott, 1982). Expert systems have 


found similar uses in many different industries, e.g. modular home building, 
telecommunications, circuit design, manufacturing, and other areas involving 
complex engineering design and manufacturing. Another long-running expert 


system (built as a case-based reasoning system; Cheetham, 2004) configures 


a color formula at GE Plastics (later SABIC) to match customers’ very 


specific requests for colors of their plastic products. 


Decision Making and Advice-Giving Tasks: Financial Decision 
Making 
The financial services industry has been a vigorous user of expert system 
techniques. Advisory programs have been created to assist bankers in 
determining whether to make loans to businesses and individuals. Insurance 
companies have used expert systems to assess the risk presented by the 
customer and to determine a price for the insurance. An important early 


application for MetLife (Glasgow, Mandell, Binney, Ghemri, & Fisher, 1997) 


analyzes thousands of insurance applications per month. In the financial 


markets foreign exchange trading is an important expert system application. 


Fraud Detection 


Some of the earliest commercial applications were in fraud detection software 
for credit card transactions (Dzierzanowski, Chrisman, MacKinnon, & Klahr, 
1989) developed for American Express. Another landmark use of expert 
systems was monitoring the huge volume of stock market transactions for the 
National Association of Stock Dealers to detect fraudulent activity (Kirkland 
et al., 1999). Criminals make use of the fact that fraudulent activity will be 
lost among the millions of financial transactions conducted daily. 
Knowledge-based systems review these transactions to separate the 
abnormal, anomalous activities from the overwhelming number that are 


normal and routine. 


Procedure and Regulation Compliance 


Another important function of an expert system is to apply relevant 
knowledge of regulations and procedures to a user’s problem, in the context 
in which the specific problem arises. Some are designed to be used by a 
diverse set of people, others by a small, restricted set. Two widely distributed 
expert systems in the former category are an advisor that counsels a user on 
appropriate grammatical usage in a text, and a tax advisor that accompanies a 
tax preparation program and advises the user on tax strategy, tactics, and 
individual tax policy. Note that in both cases the role of the system is to find 
and then present the user with knowledge relevant to a decision the user has 


to make. 


Personalized Recommendations of Products and Services 


With the growth of Internet shopping, major suppliers of goods and services 
have gathered data about shoppers’ individual preferences in order to 
recommend items that are likely to appeal to them. These systems are 
embedded within much larger shopping and ordering websites but are at work 
whenever the well-known “more like this” button appears. 

Many other examples of knowledge-based systems in actual use, and 
expert systems in particular, can be found in three decades of Proceedings of 
the Innovative Applications of Artificial Intelligence (IAAI) conference along 
with many other successful applications. The major tasks addressed by these 
systems have remained much the same over the years, at least at the high 
level of the three classes above. As new applications are featured in the 
conference, however, they frequently involve new domains of application, as 
shown in Figure 7.1, with interesting technical challenges. The first systems 
for fraud detection, for example, involved finding small variations in patterns 
within huge volumes of transaction data. A summary of technical 
developments, with additional summaries of examples, appears in Smith and 
Eckroth (2016). 


Figure 7.1 Distribution of domains in IAAI papers 1989-2016. 


Some Variations in the Implementation of Expert Systems 


One common but powerful paradigm involves chaining IF-THEN rules to 
form a line of reasoning. If the chaining starts from a set of conditions and 
moves toward some conclusion, the method is called forward chaining. If the 
conclusion is known as, say, a goal to be achieved, but the path to that 
conclusion is not known, then reasoning backwards is called for, and the 
method is backward chaining. Data-directed problem-solving is forward 
chaining; goal-directed reasoning is backward chaining. Some combination 
of forward and backward chaining is often a better strategy than either one 
alone. As mentioned, a set of explicit meta-rules can direct the selection of 
rules to be invoked depending on dynamic characteristics of each problem or 
a fixed strategy. 

In addition to the rule-based paradigm, another widely used 
representation, called the structured object (also known as frame, unit, 
schema, or list structure) is based upon a more passive view of knowledge. 
Such a unit is an assemblage of associated symbolic knowledge about an 


entity to be represented (Minsky, 1981). Typically, an object in these systems 


consists of a list of properties of the entity, associated values for those 
properties, and relationships of the entity to others; e.g. in an “IS-A” or 
“PART-OF” hierarchy. 

Since every application of a task to a domain consists of many entities 
that stand in various relations, the properties can also be used to specify 
relations, and the values of these properties are the names of other units that 
are linked according to the relations. One unit can also represent knowledge 


that is a special case of another unit, or some units can be parts of another 


unit. Structured objects are especially convenient for representing taxonomic 
knowledge and knowledge of prototypical cases. An orca, for instance, is a 
marine mammal which, in turn, is a mammal; thus orcas and all other species 
of mammals can be found to share the characteristics of mammals without 
having to duplicate the representation of those characteristics. 

The problem-solving model (or framework, problem-solving 
architecture, or paradigm) organizes and controls the steps taken to solve the 
problem. These problem-solving methods are built into program modules we 
earlier referred to as inference engines (or inference procedures) that use 
knowledge in the knowledge base to form a line of reasoning. While human 
experts probably use combinations of these, and more, expert systems have 
been successful following each of these strategies singly. 


The blackboard model of reasoning (Engelmore & Morgan, 1988; 


Erman, Hayes-Roth, Lesser, & Reddy, 1980) is effectively a dynamic mix of 
reasoning forward from available data as well as backward from overall 
goals. It is said to be opportunistic in the sense that the order of inferences in 
problem-solving is dictated by the items that seem most relevant in the 
problem description, in the partial solution, or in the knowledge base. This 
model can be used effectively to combine the judgments of multiple expert 
systems with specialized knowledge in different parts of the problem. 

Still another paradigm, which emphasizes the power of experiential 


knowledge, is case-based or analogical reasoning (Kolodner, 1993; Leake, 


1996). In a case-based reasoning system, a new problem is matched against 
previously solved cases. The closest matches are retrieved to suggest 


analogous solutions for the new problem. 


Conclusion 


Using computers to model the expertise of recognized experts has brought 
new issues into focus and has provided new experimental tools for the study 
of expertise. Each successful application confirms the adequacy of both the 
representation and inference methods for capturing the expertise needed to 
perform a specific task in a specific domain. Representing expertise about 
diagnostic problems in rules, for example, has been confirmed enough times 
that it makes sense to consider using this representation for a new 
troubleshooting problem. 

On the other hand, there can be valuable information gained when a new 
task or a new domain requires more than one of the “standard” paradigms — if 
we only collect and review these data. In one of the experiments using the 
MYCIN program for tutoring, for example, Clancey (1985) found that rules 
alone did not provide sufficient justification for their inferences. Medical 
students need to know more about the underlying reasons why a condition 
warrants an action than MYCIN’s explanations provided. 

Computational methods have added significant expressive power to the 
field of cognitive science. Mathematics is often overly precise and awkward; 
descriptions in English are often vague. Computer programming forces 
precision but also allows describing procedures within the context of 
complex situations. Expert systems, in addition, require explicit, declarative 
Statements of an expert’s knowledge and allow manipulation of that 


expertise. 


Just as human experts frequently exhibit exceptional performance in 
only one area (e.g. chess), knowledge-based systems are constrained in the 
scope of their high performance, but even more so than humans. As Davis 
(1984) and others have noted, when expert systems are given problems 
outside their assumed scope they frequently fail miserably, while human 
experts are more likely to recognize the boundaries of what they know and 
don’t know; i.e. they possess relevant meta-level knowledge (Davis, 1980). 
The resulting brittleness of knowledge-based systems makes the differences 
between the knowledge of human experts and computer programs fruitful 
areas of research in both AI and cognitive psychology. 

On the applied side, within the bounds of what expert systems do well, 
applications have been abundant and have been proliferating greatly in this 
era of ubiquitous small applications, Web apps, and smartphone or tablet 
apps. Expert knowledge has value to the millions of users of these apps and 
the consequent commercial value is driving the proliferation. They provide 
abundant data to cognitive scientists about the representation and use of 
expertise (Patel & Groen, 1991). 

Finally, one consequence of the ubiquity and value of knowledge-based 
systems, and the success of AI generally, has been societal discussion about 
the changing nature of work, the displacement of workers, and the ethics of 
letting machines make decisions for us. One pair of authors speculates 
whether the proliferation of “AI expertise” will result in a hollowing out of 
the middle class of professional workers (Brynjolfsson & McAfee, 2014). On 


a more positive side, another speculates about “a world without work” 


(Thompson, 2015) and its benefits to humankind. 
The applications of knowledge-based systems over several decades 


provide an important model for working with automation, in which Al 


intelligent assistants make human work safer, less tedious, and usually better. 
Mistakes caused by human fatigue, lack of training, or carelessness can be 
avoided when a watchful partner corrects us. Our work becomes qualitatively 


better with an intelligent assistant but it is still our work. 


References 


Ambrosino, R., & Buchanan, B. G. (1999). The use of physician domain 
knowledge to improve the learning of rule-based models for decision support. 
In Proceedings of the AMIA Annual Symposium (pp. 192-196). Washington, 
DC. 


Anderson, J. R. (1982). Acquisition of cognitive skill. Psychological Review, 
89, 369-406. 


Arocha, J. F., & Patel, V. L. (1995). Novice diagnostic reasoning in 
medicine: Accounting for evidence. Journal of the Learning Sciences, 4, 
355-384. 


Berg, C. A., & Sternberg, R. J. (1992). Adults’ conception of intelligence 
across the adult life span. Psychology and Aging, 7, 221-231. 


Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. 
Scientific American, 284 (May), 34—43. 


Bobrow, D. G., & Hayes, P. J. (1985). Artificial intelligence: Where are we? 
Artificial Intelligence, 25, 375-415. 


Boose, J. H. (1989). A survey of knowledge acquisition techniques and tools. 


Knowledge Acquisition, 1, 39-58. 


Brynjolfsson, E., & McAfee, A. (2014). The second machine age: Work, 


progress, and prosperity in a time of brilliant technologies. New York: 


Norton. 


Buchanan, B. G. (1994). The role of experimentation in artificial intelligence. 
Philosophical Transactions of the Royal Society of London A: Mathematical, 
Physical and Engineering Sciences, 349, 153-166. 


Buchanan, B. G. (1995). Verification and validation of knowledge-based 
systems: A representative bibliography. Workshop on Evaluation of 
Knowledge-Based Systems, Lister Hill Center, National Library of Medicine, 
Bethesda, MD, December. www.quasar.org/21698/tmtek/biblio.html. 


Buchanan, B. G., & Shortliffe, E. H. (eds.) (1984). Rule-based expert 
systems: The MYCIN experiments of the Stanford Heuristic Programming 
Project. Reading, MA: Addison-Wesley. 


Buchanan, B. G., Smith, D. H., White, W. C., Gritter, R. J., Feigenbaum, E. 
A., Lederberg, J., & Djerassi, C. (1976). Application of artificial intelligence 
for chemical inference XXII: Automatic rule formation in mass spectrometry 
by means of the Meta-DENDRAL program. Journal of the American 
Chemical Society, 98, 61-68. 


Buchanan, B. G., & Smith, R. G. (1988). Fundamentals of expert systems. In 
J. F. Traub, B. J. Grosz, B. W. Lampson, et al. (eds.), Annual review of 
computer science (Vol. 3, pp. 23-58). Palo Alto, CA: Annual Reviews Inc. 


Buchanan, B. G., & Wilkins, D. C. (1993). Readings in knowledge 


acquisition and learning. San Mateo, CA: Morgan Kaufmann. 


Byford, S. (2016). Google’s AlphaGo AI beats Lee Se-dol again to win Go 
series 4—1. The Verge. www.theverge.com/2016/3/15/11213518/alphago- 


deepmind-go-match-5-result. 


Chandrasekaran, B., Josephson, J. R., & Benjamins, V. R. (1999). What are 
ontologies, and why do we need them? IEEE Intelligent Systems, 14, 20-26. 


Chang, M. D., & Forbus, K. D. (2014). Using analogy to cluster hand-drawn 
sketches for sketch-based educational software. AI Magazine, 35, 76-84. 


Cheetham, W. E. (2004). Tenth anniversary of the plastics color formulation 
tool. In Proceedings of the Sixteenth Conference on Innovative Applications 
of Artificial Intelligence (IAAI-04) (pp. 770-776). San Jose, CA: Association 


for the Advancement of Artificial Intelligence. 


Chi, M., Glaser, R., & Farr, M. J. (eds.) (1988). The nature of expertise. 
Hillsdale, NJ: Erlbaum. 


Chun, H. W. C., & Suen, T. Y. T. (2014). Engineering works scheduling for 
Hong Kong’s rail network. In Proceedings of the Twenty-Sixth Conference on 
Innovative Applications of Artificial Intelligence (IAAI-14) (pp. 2890-2897). 


Quebec City: Association for the Advancement of Artificial Intelligence. 


Clancey, W. J. (1985). Heuristic classification. Artificial Intelligence, 27, 
289-350. 


CLIPS website (2004). http://clipsrules.sourceforge.net/. 


Davis, R. (1979). Interactive transfer of expertise: Acquisition of new 


inference rules. Artificial Intelligence, 12, 121-157. 


Davis, R. (1980). Meta-rules: Reasoning about control. Artificial Intelligence, 
15, 179-222. 


Davis, R. (1984). Diagnostic reasoning based on structure and behavior. 
Artificial Intelligence, 24, 347-410. 


Davis, R. (1989). Expert systems: How far can they go? Part I. Al Magazine, 
10, 61-67; Part II: AI Magazine, 10, 65—77. 


Davis, R., & King, J. (1984). The origin of rule-based systems in AI. In B. G. 
Buchanan & E. H. Shortliffe (eds.), Rule-based expert systems: The MYCIN 
experiments of the Stanford Heuristic Programming Project (pp. 29-52). 
Reading, MA: Addison-Wesley. 


Davis, R., Shrobe, H. E., & Szolovits, P. (1993). What is a knowledge 
representation? AI Magazine, 14, 17-33. 


DeepBlue (2005). www.research.ibm.com/deepblue/. 


Dzierzanowski, J. M., Chrisman, K. R., MacKinnon, G. J., & Klahr, P. 
(1989). The authorizer’s assistant: A knowledge-based credit authorization 
system for American Express. In Proceedings of the First Conference on 
Innovative Applications of Artificial Intelligence (IAAI-89) (pp. 168-172). 
Stanford, CA: Association for the Advancement of Artificial Intelligence. 


Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medial problem 
solving: An analysis of clinical reasoning. Cambridge, MA: Harvard 


University Press. 


Engelmore, R. S. (ed.) (1993). JTEC Panel on KNOWLEDGE-BASED 
SYSTEMS IN JAPAN, Distributed by National Technical Information 
Service, ISBN-10: 1883712009, ISBN-13: 978—1883712006. 


www.wtec.org/loyola//pdf/kb.pdf. 


Engelmore, R., & Morgan, T. (1988). Blackboard systems. Reading, MA: 
Addison-Wesley. 


Erman, L. D., Hayes-Roth, F., Lesser, V. R., & Reddy, D. R. (1980). The 
Hearsay II Speech Understanding System: Integrating knowledge to resolve 
uncertainty. ACM Computing Surveys, 12, 213-253. 


Feigenbaum, E. A., & Feldman, J. (1963). Computers and thought. New 
York: McGraw-Hill. 


Feigenbaum, E. A., McCorduck, P., & Nii, P. (1988). The rise of the expert 


company. New York: Times Books. 


Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). 
CogSketch: Sketch understanding for cognitive science research and for 


education. Topics in Cognitive Science, 3, 648-666. 


Forsythe, D. E., & Buchanan, B. G. (1992). Nontechnical problems in 
knowledge engineering: Implications for project management. Expert 
Systems With Applications, 5, 203-212. 


Forsythe, D. E., Osheroff, J. A., Buchanan, B. G., & Miller, R. A. (1991). 
Expanding the concept of medical information: An observational study of 


physicians’ needs. Computers and Biomedical Research, 25, 181—200. 


Glasgow, B., Mandell, A., Binney, D., Ghemri, L., & Fisher, D. (1997). 
MITA: An information-extraction approach to the analysis of free-form text 


in life insurance applications. AI Magazine, 19, 59-72. 


Goldstein, I., & Papert, S. (1977). Artificial intelligence, language, and the 
study of knowledge, Cognitive Science, 1, 84-123. 


Gordon, J., & Shortliffe, E. H. (1985). A method for managing evidential 
reasoning in a hierarchical hypothesis space. Artificial Intelligence, 26, 
323-357. 


Halevy, A. Y., Norvig, P., & Pereira, F. (2009). The unreasonable 
effectiveness of data. IEEE Expert / IEEE Intelligent Systems — EXPERT, 24, 
8-12. 


Hammond, T., & Davis, R. (2004). Automatically transforming symbolic 
shape descriptions for use in sketch recognition. In Proceedings of the 
Nineteenth National Conference on Artificial Intelligence (AAAI-04) 

(pp. 450-456). San Jose, CA: Association for the Advancement of Artificial 


Intelligence. 


Hayes, J. R. (1985). Three problems in teaching general skills. In S. F. 
Chipman, J. W. Segal, & R. Glaser (eds.), Thinking and learning skills, Vol. 
2: Research and open questions (pp. 391-405). Hillsdale, NJ: Erlbaum. 


Hayes-Roth, F., Waterman, D. A. & Lenat, D. B. (eds.) (1983). Building 
expert systems. Reading, MA: Addison-Wesley. 


Hearn, A. C. (1966). Computation of algebraic properties of elementary 
particle reactions using a digital computer. Communications of the ACM, 9, 
573-577. 


Hendler, J. A., & Feigenbaum, E. A. (2001). Knowledge is power: The 
semantic web vision. In Proceedings of the First Asia-Pacific Conference on 
Web Intelligence: Research and Development, Maebashi City, Japan 

(pp. 18—29). London: Springer Verlag. 


Hoffman, R., Baur, E., Dumer, J., Hanratty, T., & Ingham, H. (1999). 
Turbine engine diagnostics (TED). AI Magazine, 20, 69-76. 


Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus & 


Giroux. 


Kirkland, J. D., Senator, T. E., Hayden, J. J., Dybala, T., Goldberg, H. G., & 
Shyr, P. (1999). The NASD regulation Advanced-Detection System (ADS). 
AI Magazine, 20, 55-68. 


Kolodner, J. (1993). Case-based reasoning. San Mateo, CA: Morgan 


Kaufmann. 


Larkin, J.. McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and 


novice performance in solving physics problems. Science, 208, 1335-1342. 


Leake, D. B. (ed.) (1996). Case-based reasoning: Experiences, lessons, and 
future directions. Menlo Park, CA: AAAI Press/MIT Press. 


Lenat, D., & Feigenbaum, E. A. (1987). On the thresholds of knowledge. In 
Proceedings of the Tenth International Joint Conference on Artificial 
Intelligence (IJCAI-87), Milan, Italy (pp. 1173-1182). San Mateo, CA: 


Morgan Kaufmann. 


Lindsay, R. K. (2012). Understanding understanding: Natural and artificial 
intelligence. CreateSpace. ISBN-13: 978—1466450585. 


Lindsay, R. K., Buchanan, B. G., Feigenbaum, E. A., & Lederberg, J. (1980). 
Applications of artificial intelligence for chemical inference: The DENDRAL 
project. New York: McGraw-Hill. 


McDermott, J. (1982). A rule-based configurer of computer systems. 
Artificial Intelligence, 19, 39-88. 


Michie, D. (ed.) (1979). Expert systems in the micro-electronic age. 
Edinburgh University Press. 


Minsky, M. (1981). A framework for representing knowledge. In J. Haugland 
(ed.), Mind design: Philosophy, psychology, artificial intelligence 
(pp. 95-128). Montgomery, VT: Bradford Books. 


Moses, J. (1971). Symbolic integration: The stormy decade. Communications 
of the ACM, 14, 548-560. 


Moskowitz, A. J., Kuipers, B. J., & Kassirer, J. P. (1988). Dealing with 
uncertainty, risks, and tradeoffs in clinical decisions: A cognitive science 
approach. Annals of Internal Medicine, 108, 435-449. 


Motta, E. (2013). Editorial: 25 years of knowledge acquisition. International 
Journal of Human—Computer Studies, 71 (Special Issue), 131-134. 


Muratore, J. F., Heindel, T. A., Murphy, T. B., Rasmussen, A. N., & 
McFarland, R. Z. (1989). Applications of artificial intelligence to space 
shuttle mission control. In Proceedings of the First Conference on Innovative 
Applications of Artificial Intelligence (IAAI-89) (pp. 15-22). Stanford, CA: 


Association for the Advancement of Artificial Intelligence. 


Nayak, P., & Williams, B. C. (1998). Model-directed autonomous systems. 
AI Magazine, 19, 126. 


Newell, A. (1985). Artificial intelligence: Where are we? Artificial 


Intelligence, 2, 375-415. 


Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood 
Cliffs, NJ: Prentice-Hall. 


Nilsson, N. J. (1995). Eye on the prize. Al Magazine, 16, 9-17. 


O’Dell, C., & Hubert, C. (2011). The new edge in knowledge: How 
knowledge management is changing the way we do business. Hoboken, NJ: 
John Wiley. 


Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE 
Transactions on Knowledge and Data Engineering, 22, 1345-1359. 


Patel, V. L., & Groen, G. J. (1991). The general and specific nature of 
medical expertise: A critical look. In K. A. Ericsson & J. Smith (eds.), 
Toward a general theory of expertise: Prospects and limits (pp. 93-125). 
Cambridge University Press. 


Pauker, S. P., & Szolovits, P. (1977). Analyzing and simulating taking the 
history of the present illness: Context formation. In W. Schneider & A. L. 
Sagvall-Hein (eds.), IFIP Working Congress on Computational Linguistics in 
Medicine (pp. 109-118). Amsterdam: North-Holland. 


Pazzani, M. J., & Brunk, C. A. (1991). Detecting and correcting errors in 
rule-based expert systems: An integration of empirical and explanation-based 


learning. Knowledge Acquisition, 3, 157-173. 


Pearl, J. (2001). Direct and indirect effects. In Proceedings of the Seventeenth 
Conference on Uncertainty in Artificial Intelligence (pp. 411—420). San 


Francisco, CA: Morgan Kaufmann. 


Polanyi, M. (1958). Personal knowledge. University of Chicago Press. 


Polya, G. (1954). Mathematics and plausible reasoning, 2 vols. Princeton 


University Press. 


Pople, H. E., Myers, J., & Miller, R. (1975). DIALOG: A model of diagnostic 
logic for internal medicine. In Proceedings of the Fourth International Joint 
Conference on Artificial Intelligence (IJCAI-75), Tbilisi, Georgia 

(pp. 848-855). San Mateo, CA: Morgan Kaufmann. 


Rennels, G. D., Shortliffe, E. H., & Miller, P. L. (1987). Choice and 
explanation in medical management: A multiattribute model of artificial 


intelligence approaches. Medical Decision Making, 7, 22-31. 


Richards, D., & Compton, P. (1998). Taking up the situated cognition 
challenge with ripple down rules, International Journal of Human—Computer 
Studies, 49, 895-926. 


Robinson, J. A. (1968). The generalized resolution principle. In D. Michie 
(ed.), Machine Intelligence 3. Edinburgh University Press. 


Robinson, J. A., & Sibert, E. E. (1982). LOGLISP: An alternative to 
PROLOG. In J. E. Hayes, D. Michie, & Y.-H. Pao (eds.), Machine 
intelligence 10. Chichester: Ellis Horwood. 


Rulequest (2017). Rulequest editors. C5.0: An informal tutorial. 
www.rulequest.com/see5-unix.html. 


Rychtyckyj, N. (1999). DLMS: Ten years of AI for vehicle assembly process 


planning. In Proceedings of the Eleventh Conference on Innovative 


Applications of Artificial Intelligence (IAAI-99) (pp. 821-828), Orlando, FL: 


Association for the Advancement of Artificial Intelligence. 


Samuel, A. (1959). Some studies in machine learning using the game of 
checkers. IBM Journal of Research and Development, 3, 535-554. 


Scott, A. C., Clayton, J. E., & Gibson, E. L. (1991). A practical guide to 
knowledge acquisition. Boston: Addison-Wesley Longman. 


Shadbolt, N. R., & Burton, A. M. (1989). Empirical studies in knowledge 
elicitation. ACM-SIGART Bulletin (Special Issue on Knowledge Acquisition), 
108, 15-18. 


Shanteau, J. (1988). Psychological characteristics and strategies of expert 
decision makers. Acta Psychologica, 68, 203-215. 


Shaw, M. L. G., & Gaines, B. R. (1987). An interactive knowledge elicitation 
technique using personal construct technology. In A. Kidd (ed.), Knowledge 
elicitation for expert systems: A practical handbook (pp. 109-136). New 
York: Plenum Press. 


Shortliffe, E. H. (1976). Computer-based medical consultation: MYCIN. New 


York: American Elsevier. 


Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Scientist, 
621, 394—403. 


Smith, R., & Eckroth, J. (2016). Building AI applications: Yesterday, today, 
and tomorrow. AI Magazine, in press. 


Smith, R., & Farquhar, A. (2000). The road ahead for knowledge 


management: An AI perspective. AI Magazine, 21, 17-40. 


Tecuci, G., Marcu, D., Boicu, M., & Schum, D. A. (2015). Knowledge 
engineering: Building personal learning assistants for evidence-based 


reasoning. Cambridge University Press. 


Thompson, D. (2015). A world without work. The Atlantic (July-August). 


www.theatlantic.com/magazine/archive/2015/07/world-without- 
work/395294/. 


Thompson, E. D., Frolich, E., Bellows, J. C., Bassford, B. E., Skiko, E. J., & 
Fox, M. S. (2015). Process Diagnosis System (PDS): A 30 year history. In 
Proceedings of the Twenty-Seventh Conference on Innovative Applications of 
Artificial Intelligence (IAAI-13) (pp. 3928-3933). Austin, TX: Association 


for the Advancement of Artificial Intelligence. 


Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: 
Heuristics and biases. Science, 185, 1124-1131. 


Urmson, C., Baker, C., Dolan, J., Rybski, P., Salesky, B., Whittaker, W., ... 
& Darms, M. (2009). Autonomous driving in traffic: Boss and the urban 
challenge. AI Magazine, 30, 17-28. 


Weiss, S. M., Kulikowski, C. A., Amarel, S., & Safir, A. (1978). A model- 
based method for computer-aided medical decision-making. Artificial 
Intelligence, 11, 145-172. 


Wilkins, D. C., Clancey, W. J., & Buchanan, B. G. (1987). Knowledge base 
refinement by monitoring abstract control knowledge. International Journal 
of Man—Machine Studies, 27, 281-293. 


Zadeh, L. (1965). Fuzzy sets. Information and Control, 8, 338-353. 


8 


Developing Occupational 
Expertise through Everyday 
Work Activities and Interactions 


© 


Stephen Billett 
School of Education and Professional Studies, Griffith University, Australia 
Christian Harteis 
Institute of Educational Science, University of Paderborn, Germany 
Hans Gruber 
Department of Educational Science, University of Regensburg, Germany 


Prolegomenon 


Occupational expertise comprises the ability to generate goods or provide 
services with high levels of performance in both routine and non-routine 
work tasks. Such performance arises when both intra-individual (i.e. intra- 
psychological — internal capacities) and inter-individual (i.e.  inter- 
psychological — between individuals and the world beyond them) 
preconditions are met. Often, these two areas are analyzed separately, and 
from diverse theoretical standpoints. In research on expertise, the intra- 
individual view has been privileged, with the perspective of cognitive science 
focusing on the acquisition of mental structures and processes that underlie 
outstanding performance. Research that privileges contributions from outside 
the individual may focus on supra-individual entities such as teams, groups, 
communities, or even societies, or on artifacts like workplaces, professions, 
and occupations, or on situational contexts impacting how individuals are 
expected to perform and how they actually perform (Baartman & de Bruijn, 
2011; Bloor & Dawson, 1994; Dalton, Thompson, & Price, 1977). The most 


widely acknowledged approach is the one that is well represented in this 


handbook: the superior performance approach as developed by Ericsson and 
colleagues (Ericsson, Charness, Hoffman, & Feltovich, 2006). The first 
edition of this handbook gives an excellent overview of the main 
achievements of this approach and reflects the first perspective. Quite 
deliberately, a focus is set on cognitive analyses of structures and processes 
that underlie reliable outstanding performance, measured through relevant 


domain-specific tasks. Research contributing to this approach has uncovered 


many facets of the memory and knowledge of experts who achieve high-level 
performance. Strong evidence exists that such cognitive correlates of 
expertise result from extended deliberate practice (Ericsson, 2014; Ericsson, 
Krampe, & Tesch-Rémer, 1993). 

Approaches that focus on contributions outside the individual (e.g. on 
professions or occupations) aim to understand, describe, and analyze the 
successful development of such capacities in workplaces and professional 
networks. Among the constructs that may play an important role in such 
approaches are social acceptance, power, and so on. Thus, those approaches 
address whether and in what ways excellence or expert performance is 
constructed: how individuals come to learn and participate in workplace roles 
and occupational groupings. 

Of course, the underlying theoretical concepts that best describe these 
two sets of respective preconditions are not necessarily compatible. Our 
attempt here is to align and reconcile these perspectives when engaging with 
a grounded, significant, and important field of human practice: occupations. 
Too often, representatives of such research approaches are not well informed 
about — not to mention appreciative of — other approaches. Taking together 
the requirements that individuals and organizations (or groups, teams, 
workplaces, societies, etc.) have to meet to be professionally successful, we 
refer here to occupational expertise and its development. Processes associated 
with the development and maintenance of occupational expertise, in contrast 
to the “usual” acquisition of expertise, tend to happen in everyday work 
activities and interactions rather than in planned, deliberately designed 


learning and training situations (Guile & Griffiths, 2001). Nevertheless, those 


everyday or practice-based processes can also be augmented and made more 


efficacious if they are planned and supported. This relates the issues of 


occupational expertise and of deliberate practice as both are skeptical about 
institutionalized, “schooled” activities, and stress the relevance of practice- 
based learning processes. Many issues related to practice-based instructional 
support recently have been addressed in an international handbook of 
research in professional and practice-based learning (Billett, Harteis, & 
Gruber, 2014). 

In this chapter, we develop a framework to understand and analyze 
occupational expertise. We are conscious of risks associated with aligning 
different research traditions, each with their own preferred methodologies, 
scientific references, examples, and methods to assure quality of research and 
generalizability of results, and that such an alignment may not fully meet 
every single side’s expectations. Yet, the article entitled “Situated learning: 


Bridging sociocultural and cognitive theorising” (Billett, 1996) identified 


many commonalities in conceptions, focuses, and concerns, suggesting such 
an alignment was possible and worthwhile. Numerous recent attempts to 
bridge this gap have come to similar conclusions and some are referred to in 
this chapter. One bridging example is the concept of “personal professional 
theories” (Schaap, de Bruijn, van der Schaaf, & Kirschner, 2009). This 
concept is derived from the perspective of cognitivism, and is related to 


schema-based theories and models of expertise development such as 


encapsulation theory (Boshuizen & Schmidt, 1992; Schmidt & Boshuizen, 
1992) or knowledge restructuring (Boshuizen, Schmidt, Custers, & van de 
Wiel, 1995). It uses ideas such as procedural knowledge and compiled 
knowledge, and is thus consonant with a cognitive perspective. It can also be 
applied situationally and used in quite different theoretical contexts. The 
results of research on personal professional theories can, for example (we 


deliberately chose a quite controversial example), also be discussed in the 


light of cultural-historic activity theory (Engestr6m & Sannino, 2010). This 
theory offers a distinct view on the role of theory and practice and the 
interdependences of individuals and their environments, in terms both of 
organizations and of situational contexts. Activity systems, in this approach, 
are considered to be structures formed by individual practitioners, their 
communities, occupational actions, available tools and instruments, activities, 
and so forth. Many of these impacting factors help to describe the nature of 
workplaces and how they influence individuals and the relations between 
individuals. An approach that has the potential to bridge such seemingly 
irreconcilable perspectives of research as expertise theory and cultural- 
historic activity theory might be that of Social Network Analysis or 
“networked expertise” (Hakkarainen, Palonen, Paavola, & Lehtinen, 2004). 
In research about networked expertise, important relations among individuals 
and their roles in the development of expertise have also been investigated 
(Rehrl, Palonen, Lehtinen, & Gruber, 2014). These analyses of relations with 
third parties help in understanding the mechanisms that initiate and maintain 
deliberate practice activities. These mechanisms are salient when the 
attention is directed to the relation amongst individuals, co-workers, their 
trainers, coaches, and the like. Thus, more generally, the role of “persons-in- 
the-shadow” (Gruber, Lehtinen, Palonen, & Degner, 2008) has been analyzed 
to secure a broader understanding of deliberate practice and intra-individual 
cognitive development (Lehmann & Kristensen, 2014). 

Other examples of this bridging refer to the different usages of the 
concept “knowledge” itself, including the knowledge about what is not 
known or should not be done (negative knowledge: Gartmeier, Bauer, 
Gruber, & Heid, 2008). In their taxonomy, de Jong and Ferguson-Hessler 
(1996) proposed differentiating a large number of different types and 


qualities of knowledge, depending on their academic origins. In relation to 
occupational expertise, the distinction between explicit and implicit 


knowledge (Nonaka & Takeuchi, 1995) is often stressed, as is the importance 


of tacit knowledge (Eraut, 2000), particularly where a crucial quality of 
expert performance is unconscious, intuitive acting (Dreyfus & Dreyfus, 
1986; Harteis & Billett, 2013). Delineating different modes of knowledge in 
order to do justice to the complexity of workplace activities has been 
suggested. While often in research “mode-1 knowledge” is investigated 
(which is based on the canonical underlying of knowledge in relevant 
academic disciplines), workplace analyses frequently show that “mode-2 
knowledge” is more viable in real-work contexts (knowledge derived from 
research in a context of application). We consider both modes of knowledge 
as legitimate, because they both indicate success factors for occupational 
expertise and for improved professional performance. It is our long-term and 
sustained attempt to understand the requirements for occupational 
performance and its development that has led to these attempts to both draw 
upon and reconcile contributions from concepts and research traditions that 
variously privilege either the  intra-individual or  inter-individual 
contributions. These are elaborated here in consideration of developing and 


sustaining occupational expertise. 


Developing and Sustaining Occupational 
Expertise through Work Activities and 
Interactions 


Occupational expertise is central to achieving many societal and economic 
needs. These include providing the services and goods required for human 
existence and contemporary standards of living, sustenance, healthcare, 
education, and shelter, to name a few. Developing and maintaining this 
expertise is, therefore, an essential societal goal. This expertise is essential for 
individuals to secure and sustain employability across lengthening working 
lives that are increasingly subject to changing occupational and workplace- 
specific requirements. As indicated in the prolegomenon, this chapter aims to 
augment existing conceptions of expertise by elaborating the social, 
situational, and personal bases of occupational expertise and its development 
through everyday work experiences. Drawing on the heritage laid down 
through studies of expertise, it proposes that occupations have social geneses, 
are transformed by changing societal needs, yet have goals, practices, and 
outcomes that are manifested situationally. What constitutes expert 
occupational performance is held to be shaped by situational factors that 
comprise the kinds of occupational tasks, goals, and practices that are 
afforded in circumstances of the occupational practice. They reflect societal 
and situational bases that need to be mediated by and integrated into 
individuals’ personal domains of occupational knowledge by workers when 


addressing occupational tasks. 


This conception of occupational expertise and its development draws 
upon and complements conceptions that have arisen through cognitive 
science and the quest to understand what constitutes expertise within a 
domain of human practice and how this might best be developed, as proposed 
above. The premise for much of what is proposed here draws upon the 
assumption that the requirements for performance in the same occupation, 
whilst canonical, are also highly situated, that is, localized goals and 
situationally appropriate solutions. The consideration and emphasis of 
person-specific domains of occupational knowledge complement the more 
general account of research on expertise as advanced by cognitive science 
(Ericsson & Smith, 1991). We propose, ultimately, that beyond the canonical 
and_ situational, occupational knowledge is something constructed, 
represented, and enacted in person-dependent, impossible-to-codify ways by 
individuals, albeit shaped by canonical and situational premises. Hence, we 
take forward the concept of domains of knowledge and suggest that these 
exist in three ways: the canonical, situational, and personal. 

We also focus on experiences in workplaces and across working life as 
key sources of initial and ongoing development of occupational expertise. 
Certainly, there has been much and long-term interest in young people’s 
development of occupational capacities through schooling, vocational and 
tertiary education programs. Whilst experiences in educational settings are 
the subject of much research, the intentional focus here is on experiences in 
work settings and across working lives, as these are key but under-theorized 
and legitimized circumstances for learning occupational knowledge. Part of 
much contemporary occupational preparation and most of its ongoing 
development is premised on individuals’ learning through engagement in 


work activities and interactions as students or workers, not by being taught in 


educational programs (Billett et al., 2014). Such primacy of practice-based 


learning resembles the concept of deliberate practice because it requires 
intentional and considered activities aimed at the improvement of 
performance within the professional activity, regardless of its monitoring by 
institutions or the persona. Although variously explained in anthropological 
literature, through models such as knowledge encapsulation (Schmidt & 
Rikers, 2007) and acknowledged in workplace contributions to individuals’ 
cognitive adaptation and learning in those settings (Dornan & Teunissen, 
2014; Herbig & Miiller, 2014; Lehmann & Gruber, 2006), the role and 
contributions of everyday learning activities are not fully understood, 
acknowledged, or legitimated. Within contemporary “schooled societies” 
there is a tendency either to overlook or to minimize the educational worth of 
experiences outside of those in educational institutions. Here, a central 
concern is to redress that tendency. This includes offering informed accounts 
to counter those suggesting experiences are referred to as “informal” learning 
as a residual category to describe learning whose legacies are quite concrete 
and restricted to circumstances where they were learnt (Skule, 2004). 

Indeed, what constitutes occupational expertise and the contributions of 
workplace experiences to its development also questions some orthodoxies of 


educational discourses. Evidence suggests these settings support learning that 


is as rich and adaptable as any other (e.g. classrooms) (Eraut, 2004a). Work 
settings afford occupational experiences, as they require the enactment of 
activities and interactions in specific work settings (Breckwoldt, Gruber, & 
Wittmann, 2014). Hence, they can promote depth of understanding and 
honing of specific procedures, and can generate strategic procedures required 
for occupational competence (Tigelaar & van der Vleuten, 2014). The 


personal dispositions and epistemologies of individuals engaging in those 


work activities are essential here as these direct, monitor, and appraise their 
work and learning efforts (Thornton Moore, 2004). This last point is 
important as the knowledge, knowing, and performances required for 
occupational expertise are largely secured through individuals’ personal 
mediation, rather than by instruction or close guidance by others (Billett, 
2015). It follows that elaborating workplaces’ contributions to the 
development of occupational expertise necessitates going beyond the 
precepts and discourses of “schooling.” Those discourses often privilege pre- 
specified educational outcomes, teacherly practices, and activities that are 
substituted for or divorced from actual occupational practice. Increasingly, 
experiences in educational settings alone are insufficient for initially 
developing occupational capacities (Billett, 2009b). Needed now are 
conceptions of how occupational expertise can be initially developed and 
maintained through work activities and interactions (Hakkarainen, Hyt6nen, 
Lonka, & Makkonen, 2014; Rehrl et al., 2014). 

In advancing this case, occupational expertise is first delineated as 
having three bases: (a) canonical occupational knowledge, (b) situated 
manifestations, and (c) individuals’ personal representations. Canonical 
knowledge comprises societal requirements for practicing an occupation 
(what practitioners know, do, and value). This includes capacities to perform 
novel occupational tasks, taken here as key indicators of expertise, albeit in 
particular circumstances of practice (i.e. situated work requirements). 
Distinctions are then made between occupational competence (possession of 
the knowledge required for performing occupational tasks) and occupational 
expertise (being able to respond effectively to routine and non-routine 
activities in particular circumstances) that require both canonical and 


situationally pertinent capacities and solutions. These capacities are then 


proposed as being appropriated as individuals’ personal domains of 


occupational knowledge (Miller, 1996). Next, the processes for realizing and 


supporting the development of occupational expertise through work activities 
are discussed and advanced. Added here are contributions from theories that 
emphasize the social and situated contributions to learning, and of necessity 
learners’ personal epistemologies. All of this progresses on foundations laid 


by accounts within cognitive science and specific studies of expertise. 


Constituting Occupational Expertise: 
Canonical, Situational, and Personal Bases 


Occupational expertise has bases comprising canonical occupational 
knowledge, situated occupational factors, and personal representations. Each 
of these bases has implications for how occupational performance is 
conceptualized, manifested, learnt, and practiced. They are now elaborated 
through reference to and building upon what has been advanced through 


studies and contemporary accounts of expertise. 


Canonical Occupational Knowledge 


Canonical occupational knowledge comprises the conceptual, procedural, and 
dispositional capacities required to practice an occupation. Arising through 
historical and cultural need for specific capacities, this knowledge represents 
the societal expectation of what those practicing the occupation (e.g. doctors, 
teachers, and pilots) need to know, do, and value. It is expected that doctors 
will understand human physiology and diseases, can identify, select, and 
administer appropriate treatments, and can be discreet with patient 
information, for instance. Similarly, teachers are expected to be current in 
their content, effective with instruction, fair in their assessment, and able to 
exercise a duty of care toward, and be confidential about, students. Yet, 
different kinds of expectations might exist for different kinds of occupational 
practices. Whereas civilian aircraft pilots would not be expected to make 
rapid maneuvers, these are necessary for military pilots and crop dusters. We 
want cooks to be able to prepare food from diverse cuisines, not just one. 
These societal expectations are captured in textbooks, occupational 
competence standards, and requirements for certification to practice that also 
describe what is required to secure acceptance in the occupational discourses. 
They are used to select and organize educational goals and content of courses 
and to assess students’ performance (i.e. being competent) and occupational 
licensing. 

Key elements of canonical occupational knowledge are principles and 
practices that inform its enactment and adaptability, that is, informing 
occupational domain-specific principles and practices about medicine, 


aviation, teaching, cooking, and so on that can apply across the range of 


circumstances in which doctors, pilots, teachers, and cooks, for instance, 
work. Hairdressers use concepts such as the shape of clients’ faces and necks, 
and texture of their hair to make judgments about the kinds of styles to offer 
and provide for clients (Billett, 2001). Equally, there are procedures 
associated with shaping hair (such as heat, chemical, manual) that have their 
own requirements, potentials, and limits. These comprise the overall problem 
space in which their hairdressing tasks progress. Equally, informed 
procedures can assist practitioners identify and respond to particular tasks 
and problems associated with the occupation. Cooks have access to a number 
of methods of thickening sauces and will exercise each of them depending 
upon the specific dish being prepared or ingredients being used. Clinical 
reasoning, similarly, draws on a set of canonical factors associated with 
human physiology and disease (Groen & Patel, 1988) that can be adapted to 
the range of circumstances where this reasoning is required. Hence, more 
than concepts and propositions, these instances of canonical knowledge 
include bases that inform the parameters for occupational performance 
(Palonen, Boshuizen, & Lehtinen, 2014). These concepts and procedures are 
underpinned by occupational values or dispositions that are derived socially 
(e.g. pilots’ caution, teachers’ fairness, doctors’ discretion). Hence, 
developing such capacities within individuals can assist not only situated 
performance, but also applicability across settings and circumstances where 
the occupation is practiced. However, this societal account of occupational 
knowledge and particularly its representations in printed documents are 


abstracted from the actual enactment of the occupation. 


Situated Manifestations of Occupational Expertise 


Occupational expertise is manifested, enacted, and judged in specific 


circumstances of practice (Billett, 2001). The goals and requirements of each 


occupational activity are manifested in concrete situations, and the ability to 
respond effectively to them is the hallmark of occupational expertise. 
Individuals’ canonical occupational knowledge alone is insufficient as work 
performance is ultimately premised on responding effectively to these 
situationally derived goals and tasks that comprise the domain of situated 
practice. For instance, hairdressers from salons in quite different geographical 
locations, when asked to respond to the “same” hairdressing problems, 
generated responses that were subject to the exigencies of those settings 


(Billett, 2001). One salon was in an inner-city suburb and its clientele largely 


trendy young people. Another was in a low socio-economic area where 
clients wanted value for money and often complained about the haircuts and 
their cost. The third was in a rural community experiencing years of drought 
and poor returns to the community and where a trip to the hairdresser was a 
treat. The fourth was a salon whose clientele largely comprised older ladies, 
often very lonely, who would come to the hairdressing salon as much for 
social contact as to have their hair dressed. In that particular salon, knowing 
the clients’ personal histories and the names of their acquaintances was as 
important as the haircuts provided. The point is that you could not transfer 
hairdressers from across these settings and expect them to be able to respond 
to the particular requirements of those workplaces. That is, responding to 
routine and non-routine occupational tasks was, in part, a product of 


situational factors (Billett, 2001). They comprise the situated domain of 


practice that shapes and constrains the actual practice of hairdressing (i.e. 
“what we do here is”) and the problem space in which the hairdressers 
performed their occupation. 

To take another example, doctors’ roles in large metropolitan teaching 
hospitals may be quite different from those required for a nearby general 
practice, or those in hospitals in regional or remote communities. Motor 
mechanics working in roadside recovery, regular maintenance, major 
overhaul, or car racing roles require capacities to respond to the distinct tasks 
and to realize quite different outcomes (e.g. prevent a holiday from being 
ruined, recondition a vehicle, secure high levels of vehicle performance). In 
these instances, occupational performance is not only dependent on the 
canons of the occupation, but on situationally relevant capacities and 
appropriate actions. So, whereas canonical occupational knowledge informs 
actions and performance, what constitutes effective occupational performance 
is situation dependent and rendered more or less appropriate and efficacious 
by situational factors. 

It follows that as occupational performance requirements are situated 
and not uniform, expertise has to be developed and demonstrated in particular 
circumstances of practice (Billett, 2001). Hence, there is no such entity as an 
occupational expert: no expert doctor, nurse, physiotherapist, or car mechanic 
per se. Instead, the occupational expertise of doctors, nurses, 
physiotherapists, and car mechanics is premised in the circumstances of 
where they practice. Avoiding limiting occupational performance to 
particular circumstances necessitates extensive episodes of experiences that 
promote adaptability based on the canonical (i.e. the informed principles and 
practices referred to above). So, the canonical domain both grounds and 


provides the platform of concepts, practices, and dispositions that assist in 


adapting occupational knowledge to the multifold manifestations of its 
enactment. What comprises these situated practices, however, is not easily 
expressed through national occupational standards or assessed through 
standardized tests or examinations. Instead, measures of performance, 
judgments about them, and their learning are likely accessible through 
experiences in and across those settings. 

The two domains of occupational expertise referred to here and their 
attendant problem spaces are manifested in the social world, one abstracted 
from actual practice (i.e. occupational) and the other shaped by particular 
instances of practice (i.e. situational). These domains of occupational 
knowledge are what is encountered, mediated, and appropriated by 
individuals and comes to constitute their personal domains of knowledge 
comprising what individuals construct, represent, exercise, and develop 
arising from their engagement with both these canonical and situational 


bases. 


Personal Domains of Occupational Knowledge 


Individuals’ personal domains of occupational knowledge are understood in 
the widest sense as the individual and personal representation of occupational 
expertise intra-individually. They comprise conceptual, procedural, and 
dispositional occupational elements (i.e. what individuals know, can do, and 
value) that have been learnt through their experiencing, and utilized and 
developed further through their enactment. These domains are personally 
shaped and arise through and across individuals’ personal histories: their 
ontogenetic development (Scribner, 1985). They arise as a product of 
engaging with the canonical and situational bases of occupational knowledge 
but also affective and emotional bases that go beyond cognitive and codified 
aspects of knowledge. These personal domains arise from how individuals 
engage with and mediate experiences across working lives through micro- 
genetic development (moment-by-moment learning) as they think, act, and 
feel at work (Rogoff, 1990; Scribner, 1985). A highly honed form of this 


development is intuition (Harteis & Billett, 2013) arising through extensive 


experiencing, conscious consideration, and rehearsed procedures. These 
experiences shape the learning and adaptability of their occupational 
knowledge through the exercise of intentionality and agency (Goller & 


Billett, 2014) that shapes how work activities are encountered, including 


changes to work requirements. For instance, one hairdressing apprentice had 
a particular interest in hair coloring and this became important for her work, 
as one of the senior hairdressers in the salon was partially color blind. This 
led to her engaging far earlier and with greater responsibility for hair coloring 


than would be usual for apprentices (Billett, 2001). Coloring became an 


important aspect of her conception of hairdressing and identity as a 
hairdresser, and how she engaged in responding to clients’ requests for 
hairdressing. All of this shaped how she negotiated the problem space of 
assessing clients and responding to their requests, as framed by canonical 
practices and situational factors to generate problem solutions. Another 
hairdresser had an allergic reaction to the powder used in hair setting 
solutions and developed a strong disaffection for using chemicals in his 
hairdressing practice. This, likewise, led to his practice being shaped in a 
particular way and his consideration of canonical practices and situational 
requirements rendering problem solutions shaped by his personal domain of 
preferences and practices. 

So, the construction, engagement, and utilization of both the canonical 
knowledge and situational domains of knowledge both shape and are shaped 
by the personal domain of what workers know, can do, and value, and how 
confident they feel. That domain comprises the extent and diversity of their 
previous experiences (e.g. in work) that has legacies in how workers come to 
adapt their knowledge to new circumstances. This includes developing the 
kinds of informed principles and practices that underpin that adaptation, and 
the dispositions assisting in responding to those situations. Hence, 
individuals’ domains of occupational knowledge arise from what they have 
experienced across the instances of situated practice in which they have 
engaged, not just the canonical knowledge they possess (Mandl, Gruber, & 
Renkl, 1996). 

A comprehensive understanding of occupational capacities and 
occupational expertise requires the consideration of these three foundational 
domains of occupational practice, their distinct qualities and 


interrelationships. These domains have distinct epistemological dimensions 


that are central to how individuals engage in and remake occupational 
practices as work requirements change. Yet, it is individuals who ultimately 
shape their progression and the remaking of their occupational practices. This 
occurs through the moment-by-moment and day-by-day decision making by 
occupational practitioners, including engaging in the kinds of non-routine 
problem-solving tasks which are seen as the hallmark of occupational 
expertise. Hence, occupational expertise is represented in an individual way 
as a product of personal history of experiences. As learning and development 
at work vary across individuals, the individual representations of expertise 
also vary. What constitutes occupational expertise is, thus, difficult — if not 
impossible — to codify. This consideration of a personal domain of 
occupational expertise explains some of the inter-individual differences also 
found in cognitive approaches to investigating expertise that applied think- 
aloud protocols to investigate experts’ problem-solving (Schmidt, Norman, & 
Boshuizen, 1990). 

Having set out bases for what comprises occupational expertise, it is 
necessary to consider how practice-based experiences contribute to the 


development of individuals’ securing and sustaining that expertise. 


Development of Occupational Expertise 
through Work Activities 


This section discusses the process of individuals’ development of their 
personal domains of occupational knowledge that permits expertise and 
adaptability. Emphasized here are how individuals mediate what they 
experience in their everyday work activities and interactions. The discussion 
is approached, first, from a historical perspective of the development of 
occupational capacities. Then, how work activities are aligned with learning 
processes is discussed, before advancing theoretical concepts describing 
personal dimensions of knowledge as intra-mental structures and their 
learning and development that arise through everyday work activities, inter- 
mentally or inter-psychologically (i.e. between the person and the social and 


brute world beyond them). 


The Importance of Experiencing and Practice 


The task of developing occupational expertise significantly predates 
provisions of education and training programs, such as those provided 
contemporaneously by universities and technical colleges. Before the advent 
of mass education, structured programs and instruction in educational 
institutions were accessible to only a tiny minority of the population, and for 
a limited range of occupations. Across human history, the vast majority of 
occupational capacities have been learnt through participating in them. Up 
until modern times, that participation usually occurred in families or small 


local workplaces (Greinert, 2004), through active engagement in everyday 


work activities, observing and remaking what others do, experiencing success 
and failure, and extending established practices (Billett, 2014). This learning 
was largely mediated by individuals’ efforts and agency with learners 


variously being expected to steal (Marchand, 2008), apprehend (Goody, 


1989), or secure the knowledge through unobtrusive observation (Singleton, 
1989). Teaching or direct guidance by more experienced workers appears to 
have been a rarity. The former appears to be largely a product of schooling 
and the latter, it seems, only used when learning through discovery was not 
possible (Gowlland, 2012). So, across the majority of human history, 
occupations were learnt, not taught, which appears to be the case for 
sustaining occupational practices across working lives (Billett, 2014). Yet, 
many of those family businesses and cottage industries were subsequently 
displaced by industrialization. The loss of these local sites of learning led, in 
part, to the formation of mass educational provisions to meet the needs of 


industrialized economies, including preparing for new occupations required 


in newly industrial societies, and also aligning individuals’ interests to those 
of the nation state. 

There is nothing particularly novel here, as learning sciences, through 
various theoretical accounts, have long considered individuals’ learning and 
development arising mainly through their mediation of daily life experiences 


(Baldwin, 1894; Valsiner & van der Veer, 2000) rather than being taught. 


Yet, currently, much educational research and educational practice and policy 
privileges the provision of instruction, education, and training. As such, they 
risk ignoring the contributions of individuals’ engagements in practice-based 
activities and interactions. Resnick (1987) criticized the emphasis on 
instruction in ways that prompted a reconsideration of engagement in practice 
within scientific deliberations about learning and instruction. This led to fresh 
considerations of how learning in working life could inform making 
schooling experiences more effective, through processes such as reciprocal 
teaching and learning (Palincsar & Brown, 1984) and _ cognitive 
apprenticeship (Collins, Brown, & Newman, 1989) that emphasized 
positioning students as active learners, rather than being taught. Moreover, 
through government mandate, industry insistence, and community need, it 
has become almost obligatory for tertiary education provisions preparing 
students for specific occupations to include practice-based experiences in 
their curriculum. Importantly, these experiences are also helpful for 
maintaining occupational capacities across working lives. The Programme of 
International Assessment of Adult Competence data (OECD, 2013) indicates 
that workers’ engagement in both routine and non-routine problem-solving 
tasks is generative of new learning as well as refining what they know, and 


this learning arises as much through workers’ personally mediated actions as 


through guidance by supervisors and more experienced co-workers (Billett, 
2015). 

Hence, appraising the contributions of workplace experience and 
practice is central to elaborating how developing occupational expertise 
might arise through accessing and exercising occupational knowledge in 


work settings. 


Work Activities and Learning 


Each individual engagement in goal-directed activities is inevitably related 
with learning. The inference from the evidence is that engagement in familiar 
activities leads to the refining and honing of what they know, whereas 
engaging in novel activities and interactions is generative of new learning 
that is potentially adaptable to other purposes. Experiencing failure can also 
initiate learning from mistakes. The key premise of learning through and for 
work is, thus, developing knowledge by accessing workplace activities and 
interactions. This applies to individuals’ learning of both canonical and 
situational dimensions of occupational knowledge. 

Accounts of goal-directed action imply that individuals’ goal setting, 
planning and executing actions, and monitoring and evaluating their 
outcomes are central to the change or learning they encounter. These 
processes are not uniform or comprised of set sequences. Individuals may, for 
instance, define their goal after having started to act or they may modify a 
goal during acting. It is, instead, a general pattern of acting at work through 
which learning arises in personally distinct ways. Workers act on bases of 
goal-directed accomplishments, as defined and coordinated by sub-goals and 
sub-actions, through which they regulate their actions and learning 
(Hofmann, Schmeichel, & Baddeley, 2012). Multi-parted tasks might be 


broken into smaller sub-tasks that are accomplished sequentially. Yet, it is 


individuals who need to do this sequencing. Action regulation (Hacker, 2003) 
delineates various levels of mental coordination of these sub-goals and sub- 
tasks comprising (a) conscious application of declarative knowledge, (b) 


flexible action patterns (i.e. rule-based action and knowledge compilation), 


and (c) two non-conscious levels of physical skills that become automated 
(i.e. highly learnt and not requiring conscious recall) and represented in 
procedural knowledge and metacognitively as tacit or intuitive knowledge. 
Some exceptional performances (e.g. sports players and musicians) rely on 
these well-honed procedures being applied without requiring conscious recall 
(Greenwood, Davids, & Renshaw, 2012; Harteis & Billett, 2013); however, 


more conscious considered recall and monitoring of performance may be 


required for other and specific aspects of occupational practices (e.g. surgery, 
architecture, hairdressing). 

As noted, experiences in workplaces have provided essential learning 
opportunities through action (i.e. observing, imitation, and practice) and 
regulation across human history. Through observation, novices can come to 
understand and form goal states toward which their efforts and learning are 
directed. Through explanations and descriptions by co-workers, individuals 
develop declarative knowledge and approximate work tasks (Rohbanfard & 


Proteau, 2011; Rosenthal & Zimmerman, 2014). They also monitor their 


increasingly mature approximations of achieving those goal states and 
classifying declarative or procedural learning as being either positive or 
negative in terms of goal accomplishment (Gott, 1989). Positive knowledge 
is that which assists to effectively achieve those goals, whereas negative 
knowledge is that which assists to effectively avoid obstacles to achieving 
goals. Both kinds of knowledge contribute to work-related learning and 
practice-based occupational development (Gartmeier et al., 2008). Even when 
engaging in routine work activities, individuals refine and hone what they 
know and can do (Goodnow, 1986; Scribner, 1984), build links or 
associations with what they know (Groen & Patel, 1988; Wagner & 


Sternberg, 1986), and reinforce or nuance what is valued (Guberman & 


Greenfield, 1991; Perkins, Jay, & Tishman, 1993). So, even routine work 


activities and interactions provide learning experiences, making procedures 
more effective, building causal links and associations amongst concepts, and 
variously reinforcing or questioning interests and values associated with how 
and what we do (Chan, 2013; Eraut, 2004b; Kolodner, 1983; Tynjald, 2008). 


In these ways, when engaging in everyday goal-directed activities, 


workers have to decide how to act, monitor their actions, make adjustments 
or modify response, and evaluate outcomes. All of these develop their 
personal domains of occupational knowledge through situated engagement. 
What is afforded by work activities is analogous to the kinds of intentional 
experiences that teachers and educational institutions seek to enact to secure 
rich learning (Kirschner, 2002; Roth, 2001). Yet, work experiences arise 
every day across working life and are experienced by workers as variously 
being routine or novel, and together provide opportunities for generating new 
knowledge and reinforcing and refining what they already know, can do, and 
value. So as workers deploy their knowledge in everyday work activities, 
learning arises incrementally and is shaped by and contributes to their 
personal domains of occupational knowledge and, therefore, their 


competence and expertise (Billett, 2003). What is proposed here is not hybrid 


or sitting outside of existing explanatory accounts of learning or the 
development of expert performance. Indeed, a consideration of these learning 
processes is helpful in challenging some of the perhaps undeserved 


privileging of schooling processes in the development of expertise. 


Conceptualizations of Learning Processes 


Processes of skill formation have been the subject of extensive empirical 


work and theorizations. Building on Fitts (1964), Anderson (1982) proposes 


three phases of skill acquisition comprising a declarative, a compilation, and 
a tuning phase. The premise for this model is that individuals first require 
declarative knowledge (e.g. explanation from colleagues or supervisors) 
which is later proceduralized and recognized as patterns with associated 
action patterns, and finally automatized and transformed through practice to 
skilled performance, possibly supported by social partners’ environment 
through feedback and hints. With increasing levels and extent of skill 
development, progressively greater elements of this knowledge become 
proceduralized, and can be enacted without recourse to conscious regulation 
and control (Gott, 1989; Sun, Merrill, & Peterson, 2001). This is sometimes 
referred to as embodied knowledge (Reber, 1992). Ackerman (2005) offers 


an analogous account of developing skilled performance that progresses from 


controlled processing through to automatized processing as captured in a 
hierarchical three-phase model comprising a cognitive phase, an associative 
phase, and an autonomous phase of skill acquisition. Learning in the 
cognitive phase is premised on individuals’ awareness or engagement when 
engaging in novel activities and interactions. They need to make sense of the 
situation or instruction, identify and construe relevant goals, develop 
appropriate responses, and realize the goals to which their efforts are 
directed. Hence, individuals’ thinking, acting, and learning are mediated by 
the degree of conscious engagement when undertaking work tasks. In the 


associative phase, appropriate strategies are being proceduralized, in ways 


consonant with what Anderson (1982) proposed. As performance increases 
with speed of perception, compilation of separate specific procedures into 
sets of compiled procedures occurs which are enacted with minimal reliance 
on conscious memory (Sun et al., 2001). This development permits 
individuals’ conscious memory to be deployed in monitoring and evaluating 
their actions, thereby securing the higher level outcomes that enable 
undertaking work activities efficaciously or strategically (Kirschner, 2002). 
Those skills, finally, become automatized in the autonomous phase, so they 
require little conscious attention to be enacted. Consequently, frequently 
encountered tasks or interactions do not require conscious cognitive 
engagement. Instead, because of what individuals now know, can do, and 
value, those activities and the learning arising from them can be initiated and 
progressed at the associative or even autonomous phase. 

Activities and interactions in work settings, therefore, enable the 
progressive modification of individuals’ information processing from being 
controlled, aware, and demanding, to being automatized and compiled and 
able to act increasingly autonomously. This development is important for 
work activities as processing occurs quickly and requires limited cognitive 
resources, which can be used to monitor and evaluate progress. However, it is 
necessarily based upon extensive rehearsal, and particular work activities and 
interactions likely play key roles in this learning process and are shaped by 
situationally derived experiences (Reber, 1989). Familiar experiences permit 
honing and linking of concepts and procedures, yet information processing 
may remain in the declarative phase if too many novel experiences are 
encountered. Addressing novel activities or interactions requires effortful 
conscious thought and processing information. Importantly, the degree of 


novelty is person dependent, because of individuals’ prior experiences and 


experiencing (Billett, 2013; Valsiner, 2000). What for one person is a highly 


novel experience is routine for another. So, although it is helpful to identify 
the pedagogic potential of particular work activities and interactions, it is not 
possible to confidently predict actual learning that will arise because of their 
person dependence. These accounts of skill formation concur in proposing 
that through repertoires of experiences, changes to knowledge structures 
move from a reliance on effortful processing to circumstances that permit 
application without as much cognitive demand (Kirschner, 2002). They are 
also analogous to those processes that refer to conceptual development and 
how propositions, causal links, and associations arise through episodes of 
experiences (Groen & Patel, 1988; Shuell, 1990). So, whilst these accounts 
advise the processes of conceptual and procedural development that are 
central to occupational expertise, they leave open questions associated with 
how everyday work activities can best guide, support, and augment the 


development of occupational expertise. 


Supporting the Development of Occupational 
Expertise through Work Experiences 


Despite a reliance on everyday experiences in work settings across much of 
human history, alone they may be insufficient to develop the requirements for 


contemporary occupational capacities (Elmore & Massey, 2012). Those 


limitations are at their greatest for those who are newcomers and when the 
learning required is difficult to secure through observation and imitation 
(mimesis). Rhymes and mnemonics were used by artisans as long ago as in 
early imperial China to assist the development of skills for manufacturing 


porcelain, for instance (Billett, 2014). Although the evidence of direct 


instruction or teaching in the workplace is limited across much of this history, 
some forms of direct guidance are evident. For instance, evidence exists of 
experienced artisans placing their hands upon those of novice potters to assist 
them develop the haptic skills required for shaping clay on the potter’s wheel 
(Gowlland, 2012; Singleton, 1989). Similarly, Pelissier (1991) reports that 


beach debris was used as artifacts to assist Micronesian fishermen in learning 


to recognize the star pattern to navigate with at night. Yet, in contemporary 
times, the growing reliance on symbolic and conceptual knowledge in 
technologies that are part of work practices (e.g. computers, principles for 
practices, factors, etc.) are not always easily experienced or learnt (Martin & 


Scribner, 1991). Accessing and learning this kind of knowledge may need to 


be supported and augmented by interactions with more experienced 
practitioners, yet still is ultimately mediated and learnt by individuals 


themselves. 


Just as with provisions of experiences for students in educational 
institutions, when considering how the learning of these kinds of knowledge 
might be best supported it may be helpful for these experiences to be ordered 
and augmented in some way. Identified here are three broad dimensions to 
this support and augmentation. First is the organization, ordering, and 
sequencing of workplace experiences to assist the incremental development 
of occupational capacities: the practice curriculum. Second, there are 
pedagogic practices enacted as part of everyday work activities that can assist 
in securing access to and the learning of conceptual and procedural 
occupational knowledge, and occupationally appropriate dispositions. These 
pedagogic practices comprise interpersonal processes such as guided learning 
strategies, apprenticeship interventions, and access to mobile technologies 
that provide access to conceptual constructs. Other pedagogic practices are 
more founded in workplace interactions, including the use of narratives, 
storytelling, half-completed jobs, joint problem-solving and engagement with 
heuristics, and most importantly work activities that are inherently 
pedagogically rich. These go beyond classroom-like pedagogic practices and 
embrace those more suited to the circumstances of practice. The third 
dimension is what engages and directs learners’ intentionalities, interests, and 
capacities (Malle, Moses, & Baldwin, 2001) to promote rich learning (e.g. 
deliberate practice, mimetic learning, supervision, feedback), referred to here 
as personal epistemologies: how individuals elect to engage and direct them. 

In the sections below, these means of promoting and building expertise 


are discussed and illustrated. 


Practice Curriculum 


The word curriculum originally meant “the course to follow” or “the path to 
progress along” (Marsh, 2004). Analogously, the concept of practice 
curriculum is premised on learners’ progression through activities in a 
particular work setting. There is often a deliberate structuring of experiences 
to assist incremental progression through and engagement in occupational 
activities through which novices learn. The anthropological and historical 
literature suggests that these experiences are afforded through activities and 
interactions in both (a) the lived experience of occupations being enacted, and 
(b) the deliberate arrangements to access and learn the occupational activities 


and knowledge that otherwise might be inaccessible (Bunn, 1999; Jordan, 


1989; Lave, 1990; Rogoff, 1990). So the practice curriculum encompasses 
participation in the everyday activities and interactions of particular 
circumstances of practice and also intentional arrangements to develop both 


canonical competence and situation expertise. 


The lived experience of occupations being enacted 


Lave (1990) identified apprentices’ learning of tailoring progressing through 
being immersed in the practice of tailoring, and progressing through a 
sequence of tasks. This sequencing and progression was based on 
apprentices’ learning, unaccompanied by any direct guidance. The 
apprentices observed tailors and their working and used artifacts, such as 
completed garments or those under construction, to guide their own 
approximations of achieving observed performances. They gradually 


progressed along a pathway of tailoring activities on the garments being 


made in the workshops. This pathway involved initially engaging in tasks in 
which errors could be tolerated, and progressed by engaging incrementally in 
more demanding tasks that were commensurate with their developing 
tailoring competence (i.e. where their errors would not jeopardize the 
garments they were making). Billett (2006) identified analogous sequencing 
of activities in the example of hairdressing salons mentioned earlier, in which 
hairdressers also progressively engaged in tasks commensurate with their 
developing occupational competence, but founded on experiences that 
progressively developed capacities, such as learning to communicate and 
negotiate with clients, washing hair, shaping and then learning to cut it — 
tasks that are canonical to hairdressing. Similar patterns of progression were 
also identified in food manufacturing and for hotel room attendants (Billett, 
2011). In essence, there were courses to follow — tracks to progress along 
shaped by workplaces’ productive requirements — and through participating 
in their enactment, learning arose. The kinds of opportunities afforded in such 
settings shape the learning of occupational practice through the learners’ 
engagement with and mediation of it. Whilst permitting participation and 
providing opportunities for observation, imitation, feedback, and practice, 
they are also shaped by particular work requirements. So, more than 
developing canonical occupational knowledge, they can lead to developing 
situational expertise. While this is very helpful, these experiences and 


learning need to be augmented so that adaptable learning also arises. 


Providing access to knowledge that might not otherwise be learnt 


Work activities can provide access to knowledge that might not otherwise be 


accessible or learnt through the lived experience of workplaces. For instance, 


Marchand (2008) refers to apprentice minaret builders engaging in intentional 
structured learning experiences, commencing in mixing mortar and 
fashioning stones prior to supplying stone masons with these materials in 
ways that permitted them to observe and, ultimately, practice stone-laying on 
the inside of a minaret being built from the inside out. Only when they had 
honed these skills were apprentices permitted to place stones on its outside: a 
critical task and one crucial to the finished appearance of the minaret. 
Singleton (1989) also identified stages through which apprentices progress in 
a Japanese pottery workshop, premised on access to the potter’s wheel. The 
Stages are: (a) pre-practice observation with apprentices engaged in menial 
work and household tasks, (b) tentative experiments at the wheel outside of 
work time, (c) assigned regular practice at the wheel, (d) assigned production 
at the wheel, and (e) a period of subsequent work in the shop to repay the 
training. So, these occupations have requirements that need to be met through 
structured engagement in work activities that also have inherent pedagogic 
potential, including making the knowledge required for performance 
accessible. 

Such curricula or pathways as means of organizing learning are not 
restricted to occupations that are held to be low status and of relatively low 
skills. Junior doctors also have a set of experiences through which they 
progress in hospitals in learning to practice medicine effectively (Sinclair, 
1997). Initially, they might engage in the admission and examination of new 
patients, sometimes repeating what has been done by the admitting doctor. 
Having learnt and honed these skills, they progress to other kinds of 
activities, building upon foundational capacities of understanding patients 
and diagnosing their conditions. Similarly, Jordan (1989) identified the 


ordering of skill acquisition of Mexican birth attendants. They moved 


through phases of practice associated with developments within the prenatal 
phase, then on to the birthing process and postnatal support, each of which 
has crucial elements. It is noteworthy that across these various models of 
practice curricula (i.e. the lived experience and intentionally organized ones), 
the nature and ordering of experiences provide opportunities to engage 
progressively with and learn in a specific instance of occupational practice. 
This includes gaining access to the occupational goal states: what has to be 
achieved for effective practice in that particular setting. 

In summary, developing canonical occupational competence and 
situated expertise requires, first, engagement in the lived experience of work 
over time and through authentic work activities; second, progressing along a 
pathway of activities exposing individuals to goals for effective work, and 
progressively accessing experiences that incrementally develop 
understanding, values, and procedures required for effective work 
performance; third, providing access to particular kinds of experiences to 
develop capacities that will not be learnt through engaging in the lived 
experience of the workplace alone; and, fourth, ordering and sequencing of 
experiences to most effectively support individuals’ learning. Yet, as 
foreshadowed, it is necessary to enrich those experiences so they can be more 
pedagogically effective, and also to develop adaptable understandings and 
procedures. Indeed, progression along these pathways is also premised on 
two additional factors. The first is the provision of experiences that augment 
the workplace experiences; the second, how learners engage in and mediate 


what they experience. 


Practice Pedagogies 


Practice pedagogies are means by which leaming through everyday work 
activities and interactions can be supported and/or augmented. These include 
making accessible the occupational knowledge needing to be learnt for 
canonical competence, and situated expertise which includes developing 
adaptable capacities that might not be learnt through individuals’ mediation 
of those experiences alone, that is, learning through discovery. As 
occupational knowledge arises in the social world, means of accessing that 
knowledge can be essential. The augmentation extends to assistance in 
making links, propositions, and causal associations amongst concepts that 
characterize depth of knowledge that is central to occupational expertise; and 
also means for promoting procedural and dispositional occupational 
knowledge and its application to new circumstances and problems. 

Practice pedagogies are usually distinct from those enacted in 
classrooms. They primarily are deployed whilst engaging in work tasks. They 
can comprise particular kinds of engagements with others, but also with 
artifacts, objects, and interactions and guidance with others. As noted, 
Pelissier (1991) identified where artifacts and guidance were used 
intentionally to assist learning. Importantly, some activities are potentially 
pedagogically rich, because of inherent qualities in promoting learning 
through coming to know (i.e. construal of what is experienced), engaging in, 
and evaluating actions and responses (i.e. constructing knowledge micro- 
genetically). Nurses’ handovers can provide opportunities for developing 
deep understanding about patient care, as they engage nurses in verbalizing 


their understandings and involve discussing (a) the patients, (b) their 


condition(s), (c) their treatment, (d) their progress with that treatment, and (e) 
formulation of prognoses (Newton, Billett, Jolly, & Ockerby, 2011). These 
five interrelated considerations afford contextually rich bases for learning 
about patient care, including securing factual knowledge, enriching concepts, 
and building of causal links and propositional associations amongst concepts 
through these discussions that also assist the formation of goals for nursing 
activities. Doctors’ mortality and morbidity meetings perform similar roles, 
with a particular emphasis on diagnosing problems arising through particular 
critical medical cases. In both examples, those participating in these work 
activities have to engage in, follow, and evaluate what is being presented and 
discussed, and make judgments about the processes and outcomes. These 
experiences support learning of robust (i.e. adaptable) procedural knowledge 
and depth of propositional knowledge of the kinds required for expert 
performance. 

The same applies for close or proximal interactions with more 
experienced interlocutors (Rogoff, 1990, 1995) and forms of close or 
interpersonal guidance, such as scaffolding, modeling, coaching (Collins et 


al., 1989), and guided learning (Billett, 2000). These practice pedagogies 


exemplify learning being guided by more expert partners who can assist the 
development of less-experienced individuals through joint problem-solving. 
More experienced workers can advise about tricks of the trade or heuristics, 
and assist novices develop schemas for undertaking work tasks, including 
heuristics and mnemonics (Rice, 2010). Mnemonics can comprise letters to 
remind doctors about a series of interrelated conditions (e.g. the causes of the 
distended abdomen can be remembered as the Five Fs — fat, fluid, flatus, 
faeces, and foetus). Junior doctors are advised to remember particular 


illnesses or diseases by referencing to patients in which they first identified 


them (e.g. remember Mr. Taylor), as this assists recalling characteristics of 
the disease for later diagnoses (Sinclair, 1997) and developing “illness 
scripts” (Schmidt & Boshuizen, 1993). These pedagogies require learners’ 
engagement and rehearsal to prompt their recall. These qualities are 
generative of what Barsalou (2003) refers to as simulation: the multimodal 
and sensory representation of knowledge that permits recall and applicability. 
They are also the kinds of development that permit adaptability to new 
circumstances. 

In sum, practice pedagogies play particular and salient roles in 
augmenting workplace learning experiences. These include: (a) making 
accessible knowledge that otherwise might not be accessed or learnt; (b) 
identifying and providing opportunities to practice and be guided in honing 
and refining what is being learnt; (c) supporting the development of 
knowledge through indexing it to practices that have implications for 
adapting it to new circumstances and tasks; (d) promoting recall or utilization 
through specific strategies and processes; and (e) providing access to artifacts 


and activities that support individuals’ learning. 


Personal Epistemologies 


Across this chapter, personal epistemologies and epistemological acts have 
been emphasized as key mediating means and contributors to individuals’ 
process of experiencing and construction of their personal dimension of 
occupational competence and situated expertise (e.g. observing and imitation, 
listening, engaging in tasks, and deliberate practice progresses) (Billett, 
2009a). Key premises for developing occupational competence and expertise 
associated with these epistemologies include (a) individuals indicating an 


interest and willingness to learn occupations (Bunn, 1999; Singleton, 1989); 


(b) learning to engage effectively in developing occupational capacities 


(Gowlland, 2012; Marchand, 2008); and (c) actively engaging in learning 
through work (Rice, 2010). Therefore, these epistemologies both assist and 
direct individuals’ capacities and embodied knowledge, their sense of self 
(i.e. subjectivity) and gaze (i.e. how they view the world and how they 


believe the world is viewing them) (Davies, 2000), and reciprocally shape the 


direction and intentionality of their efforts (Zimmerman, 2006). All of these 
premises are particularly relevant for constructing the “hard-to-learn” 
knowledge necessary for both occupational competence and _ situational 
expertise as it requires learners’ effortful and intentional participation and 
willingness to engage interdependently with others and artifacts. 

As discussed above, through such activities arises important embodied 
knowledge (Jordan, 1989; Lakoff & Johnson, 1999; Reber, 1992) or 


processes of proceduralization (Anderson, 1982) comprising the compilation 


and automatization of knowledge. This is the case whether referring to haptic 


or other sensory forms of knowing, for example, somatic markers (Damasio, 


2012) that all contribute to expert performance. Personal epistemological 
practices are, therefore, central to how learning progresses in and through 
occupational practice. For example, it is impossible to learn bicycling without 
practicing and experiencing the problem of balancing. Ultimately, the active 
engagement and construction of knowledge identified widely across this 
literature (Marchand, 2008; Singleton, 1989; Webb, 1999) is the key premise 


which, as Kosslyn, Thompson, and Ganis (2006) note, includes individuals 


having to assent to engage in and learn the occupational practice. Specific 
epistemological processes, such as ontogenetic ritualization (Tomasello, 
2004) — negotiating with a social partner to secure access — are necessary to 
gain access to the knowledge required for work. These all assist in explaining 
the person-dependent process of learning and developing domains of 
occupational knowledge comprising both its canonical and _ situational 
manifestations, albeit largely through mediating experiences. 

In summary, individuals’ personal epistemologies are central to learning 
through practice and essential for rich learning from _practice-based 
experiences and pedagogic practices. They include (a) an active interest and 
engagement in work-related learning; (b) readiness in terms of interest and 
knowing how to be positioned as effective learners; (c) engaging and learning 
interdependently in practice settings and through activities and interactions; 
(d) developing capacities to come to know, including haptic, auditory, 
sensory, and procedural capacities; and (e) engaging with others and artifacts 
to actively access understandings, values, and procedures; (f) and in ways 


that can adapt to other circumstances. 


Developing Occupational Expertise through 
Everyday Work Activities and Interactions 


The approach advanced in this chapter offers a view on occupational 
expertise that acknowledges the well-established cognitive accounts on the 
one hand, but also complements this perspective, on the other hand, by 
exploring accounts that consider the contributions of the social and cultural 
environment for the development and exercise of occupational expertise. 
Hence, this chapter merges two distinct academic perspectives that are more 
or less isolated from each other. The chapter has elaborated how the social 
and cultural world shapes the kind of activities humans engage in, how they 
engage and the consequences of that engagement in terms of their learning 
and development. Such a perspective reveals that many of the requirements 
for occupational expertise, including adapting occupational capacities to new 
circumstances and tasks, arise through everyday workplace activities and 
interactions. There is no contradiction between cognitive and socio-cultural 
accounts of occupational expertise that are adopted here. Instead, they differ 
in their focuses on individuals or on the socio-cultural environments. 
Differences arise, however, in conclusions about how best to support the 
development of expertise. A cognitive perspective may lead to a focus on 
instruction whereas the socio-cultural perspective emphasizes the importance 
of social negotiations and practices for development. A view into the past 
reveals that learning through practice has been the basis through which most 
occupational capacities have been developed across human history and, 


likely, across working lives. Yet, to more effectively secure those capacities 


and contemporary occupational expertise, the organization of workplace 
experiences (i.e. practice curriculum) and their augmentation through practice 
pedagogies are required. The chapter discussed the central role of learners’ 
personal epistemologies and how they mediate the creation and exercise of 
personal domains of occupational knowledge. A core argument is that they 
are shaped by what comprises canonical occupational knowledge and situated 
requirements for performance. Importantly and inevitably, learning as 
described here is informed and shaped by the social and material work 
environment. However, this kind of learning through work cannot be taken 
for granted, because that learning sometimes may come at a cost to 
immediate production or provision of services. But in many ways, learning 
through work is essential for the workplace, its efficacy and continuity. 
Indeed, to deny the contributions of work activities to learning risks 
inhibiting workplace developments and erecting barriers to learning. Workers 
may come to perceive their learning as being unappreciated with the potential 
consequence that they will not seek out learning opportunities or engage in 
intentional learning activities. Hence, optimizing learning through everyday 
work activities is far from a given as these activities usually have to follow 
the imperative of efficiency — and learning efforts may impede efficiency. 

By drawing upon accounts from cognitive science, and historical and 
anthropological studies, what has been proposed here aims to offer 
explanatory accounts of both occupational competence and situated, yet 
adaptable, expert performance and also how these arise through practice 
settings. In doing so, this chapter has sought to align and reconcile 
contributions across a number of disciplinary fields to inform the important 
task of identifying qualities of expert occupational practice and how it might 


be developed. That alignment and reconciliation stands to contribute to the 


work over the past two decades by those who are interested in integrating 
these contributions in addressing important societal issues rather than seeking 


to differentiate on the basis of the disciplinary origins of their contributions. 
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Introduction 


From a social perspective, the “expert” is an ascription. The kernel or 
expected value attached to this ascription is the specific knowledge we might 
share, or specific service we might receive, from the expert. The criteria for 
considering someone an “expert” vary, from qualifications (e.g. “certified 
public accountant”), proven experience (e.g. “10 years’ experience in 
turnaround management”), or demonstrated performance (e.g. “one of the 
top-ten tennis players in the world”) to roles within an organization (e.g. 
“responsible for our business in Asia”). Depending on the context, we speak 
of “expert drivers” as well as “court-appointed experts” or simply “the 
experts.” Thus, expert status might refer to an “elite” and indicate something 
exclusive (e.g. Nobel Laureates); it can be relative, as in the case of the 
expert driver; and might even be transitory, such as in the case of some 
“experts” on TV game shows. 

From a historical point of view, we see various predecessors of modern 
experts. For instance, we can conceive of priests or shamans as an extreme, 
undifferentiated version of “experts” in premodern societies, encompassing 
the roles of counsel, physician, and medium. In the rising empires of 
antiquity we see the growing importance of scholar-officials — experts with 
literacy skills — such as the Chinese mandarins, often charged with extended 
official duties in astronomy, architecture, or bureaucratic planning. Schools 
were founded to systematically prepare these scholars. In medieval times, 
merchants and artisans (bakers, shoemakers, carpenters, etc.) formed trade 


guilds that controlled quality standards, prices, and the rules for 


apprenticeship, thereby organizing the work and markets for craftsmanship in 
European cities. The historical view shows two trends: differentiation and 
(self-)organization. Guilds were self-organized and can be considered the 
predecessors of today’s professions. Knowledge and services also became 
more specific: in medieval Nuremberg, we see more than a dozen guilds 


solely for metalworking (cf. Braudel, 1992), and today there remain more 


than 100 ancient and modern guilds (“livery companies”) listed by the City of 
London (City of London, 2016). 

Our chapter consists of three parts that draw from different disciplines. 
From a sociological point of view, expertise has long been professionalized, 
thus experts are professionals: “Professionalism has been the main way of 


institutionalizing expertise in industrialized countries” (Abbott, 1988, p. 323). 


Consequently, the first and main part of this chapter introduces 
professionalism as an organized and differentiated form of knowledge-based 
work in current societies. In the second part, we focus on science as the main 
reference system for knowledge, e.g. for professional knowledge, accepted in 
current societies. Drawing from the history of science, we depict the 
evolution of the role of scientists, which itself became professionalized in the 
twentieth century. In the third part, we abstract from professionals and 
scientists, and reflect on expert roles in general, based on research from social 
psychology. This results in thinking about relative — even everyday — experts 
(beyond professions and science), as well as rethinking expertise as a 
professionalized competence in differentiated domains that display more or 


less (self-)organization of experts and their communities. 


Professionalism: The Sociology of 
Professional Groups 


One way of operationalizing and analyzing the concept of expertise 
sociologically is by means of its formation and utilization in different 
professional occupational groups. Consequently, this first part of the chapter 
focuses on the history, concepts, and theories of the sociology of professional 
groups. This intellectual field has a long and complex history. It is clearly 
linked — and closely associated with — the sociologies of work and 
occupation, where initially Anglo-American sociologists began to 
differentiate particular occupations (such as law and medicine) in terms of 
their aspects of service orientation and “moral community” and hence their 


contribution to the stability and civility of social systems. 


The concept of profession is much disputed (Adams, 2015; Evetts, 2013; 
Sciulli, 2005). During the 1950s and 1960s, researchers shifted the focus of 


analysis to the concept of profession as a particular kind of occupation, or an 


institution with special characteristics. The difficulties of defining the special 
characteristics and clarifying the differences among professions and other 
(expert) occupations troubled analysts and researchers during this period 


(such as Etzioni, 1969; Wilensky, 1964). It is generally the case, however, 


that precision about what is a profession is now regarded more as a diversion 
in that it did nothing to assist understanding of the power of particular 
occupational groups (such as law and medicine, historically) or of the 


contemporary appeal of the discourse of professionalism in all occupations. 


In the following subsections, we will provide a historical account of the 
sociology of professional groups. We then expand on the current “discourse 
of professionalism” that pervades occupational work contexts. The third 
subsection discusses how the discourse of professionalism is used to control 
work. The final subsection builds a bridge to psychology, and discusses the 


epistemology of professional work. 


The Early Years: From Professions to Professionalization 


The earliest analyses and interpretations of professional groups tended to 
focus on and utilize the concept of professionalism. For the most part these 
analyses referred to professionalism as providing a normative value and 
emphasized its meanings and functions for the stability and civility of social 
systems. Durkheim (1992) assessed professionalism as a form of moral 
community based on occupational membership. Tawney (1921) conceived 
professionalism as a force capable of subjecting individuals to the needs of 
the community. Carr-Saunders and Wilson (1933) saw professionalism as a 
force for stability and freedom against the threat of encroaching industrial 
and governmental bureaucracies. Marshall (1950) emphasized altruism or the 
“service” orientation of professionalism and how professionalism might form 
a bulwark against threats to stable democratic processes. 

The best known, though perhaps the most frequently misquoted, attempt 
to clarify the special characteristics of professionalism and its central 
normative and functional values was that of Parsons (1951). Parsons was one 
of the first theorists to show how the capitalist economy, the rational-legal 


social order (cf. Weber, 1979), and the modern professions were all 


interrelated and mutually balancing in the maintenance and stability of a 
fragile normative social order. He demonstrated how the authority of the 
professions and of bureaucratic organizations rested on the same principles 
(for example, of functional specificity, restriction of the power domain, 
application of universalistic, impersonal standards). The professions, 


however, by means of their collegial organization and shared identity, 


demonstrated an alternative approach to the hierarchy of bureaucratic 
organizations, toward the shared normative end. 

While Parsons distinguished between professions and occupations, 
Hughes (1958) regarded the differences between professions and occupations 
as differences of degree rather than kind, in that all occupational workers 
have expertise. The focus thus shifted to the relationships and interactions 
between the professions and their clients. Hughes used the expression: 
“professionals profess” (Hughes, 1965, p. 2). From this point of view, 
professionalism implies the importance of expertise but also of trust in 
economic relations in societies with an advanced division of labor. In other 
words, laypeople must place their trust in professional workers (electricians 
and plumbers as well as lawyers and doctors) and, as a result, some 
professionals acquire confidential information. Professionalism requires 
professionals to be worthy of that trust, that is, to maintain confidentiality and 
to protect private information and not exploit it for self-serving purposes. In 
return for this professionalism in relations with clients, professionals are 
granted authority, rewards, and high status. 

During the 1970s and 1980s, when sociological analysis of professions 
was dominated by various forms of professionalism as ideological theorizing, 
and under the influence of Marxist interpretations, a concept that became 
prominent was that of the professional project. The concept was developed 
by Larson (1977) and included a detailed historical account of the processes 
and developments whereby a distinct occupational group sought a monopoly 
in the market for its service and, in addition, achieved status and upward 
mobility (collective as well as individual) in the social order. The idea of a 
professional project was developed in a different way by Abbott (1988) who 


examined the carving out and maintenance of a jurisdiction through 


competition within “the system of professions,” as well as the requisite 
cultural and other work that was necessary to establish the legitimacy of a 
monopoly practice. Larson’s work is still frequently cited, and MacDonald’s 
textbook on professions (1995) continues to support Larson’s analysis in the 
examination of the professional field of accountancy. The outcome of the 
successful professional project was a “monopoly of competence legitimized 
by officially sanctioned ‘expertise,’ and a monopoly of credibility with the 
public” (Larson, 1977, p. 38). 

Another version of the “professionalization as market closure” idea has 
been the notion of professions as powerful occupational groups who not only 
close markets and dominate and control other related occupations, but who 
can also “capture” states and negotiate “regulative bargains” (Cooper, Lowe, 


Puxty, Robson, & Willmott, 1988) with states in the interests of their own 


practitioners. Again this was an aspect of theorizing about professions in 
Anglo-American scholarship that began in the 1970s (e.g. Johnson, 1972), 
was influenced by Marxist interpretations, and focused on medicine and law. 
It has been a particular feature of analyses of the medical profession (e.g. 
Larkin, 1983) where researchers interpreted relations among health 
professionals as an aspect of medical dominance as well as gender relations 
(e.g. Davies, 1995). In a more recent, neo-Weberian interpretation by Saks 
(2010), professional groups are conceptualized in terms of “exclusionary 
social closure in the marketplace, sanctioned by the state” (p. 887). In that 
line, Faulconbridge and Muzio (2011) propose a “transnational sociology of 
the professions” to take into account “supra-national mobility and 
connections of professionals, their work, clients and governmental actors” (p. 
237), 


The Discourse of Professionalism 


In current work and employment contexts (such as professional work in 
organizations) it is the increased use of the discourse of professionalism in a 
wide range of occupational workplaces that is important and in need of 
further analysis and understanding. The discourse of professionalism is used 
as a marketing slogan (e.g. “have the job done by professionals”) and in 
advertising to attract new recruits (e.g. “join the professionals” — the army) as 
well as customers (Fournier, 1999). It is used in occupational recruitment 
campaigns, in company mission statements, and in expressions of 
organizational aims and objectives to motivate employees. The discourse of 
professionalism has entered the managerial literature and has been embodied 
in training manuals. Even occupational regulation and control (both internal 
and external forms) are now explained and justified as means to improve 
professionalism in work. The concept of professionalism has an appeal to and 
for practitioners, employees, and managers in the development and 
maintenance of work identities, career decisions, and senses of self. 


The claim is now being made (e.g. Freidson, 2001) that professionalism 


is a unique form of occupational control of work, which has distinct 
advantages over market, organizational, and bureaucratic forms of control. In 
assessing the political, economic, and ideological forces that are exerting 
enormous pressure on the professions today, Freidson (1994) defended 
professionalism as a desirable way of providing complex, discretionary 
services to the public. He argued that market-based or organizational and 
bureaucratic methods impoverish and standardize the quality of service to 


consumers and provide disincentives to practitioners. Thus, professions might 


need to close markets in order to be able to endorse and guarantee the 
education, training, expertise, and tacit knowledge of licensed practitioners, 
but once achieved, the profession might then concentrate more fully on 
developing service-orientated and performance-related aspects (Halliday, 
1987). The process of occupational closure will also result in monopoly in the 
supply of the expertise and the service, and probably also lead to privileged 
access to salary and status. However, as has been noted, the pursuit of private 
interests is not always in opposition to the pursuit of the public interest, and 
indeed both can be developed simultaneously (Saks, 1995). 

The comparative work of Hughes (1958), and his linking of professions 
and occupations, also constitutes the starting point for many micro-level 
ethnographic studies of professional socialization in workplaces (e.g. 
hospitals and schools) and the development in workers of shared professional 
values and identities. This shared professional identity (which has been a 
major research focus for French researchers; e.g. Dubar, 2000) is associated 
with a sense of common experiences, understandings, and expertise, shared 
ways of perceiving problems and their possible solutions. This common 
identity is produced and reproduced through occupational and professional 
socialization by means of shared educational backgrounds, professional 
training, and vocational experiences, and by membership of professional 
associations (local, regional, national, and international) and institutes where 
practitioners develop and maintain shared work cultures and common values 
(e.g. Adler, Kwon, & Heckscher, 2008; Evetts, 2011). 


Today, for most researchers, professions are regarded as essentially the 


knowledge-based category of service occupations which usually follow a 
period of tertiary education and vocational training and experience (cf. 
Brante, 2010; Evetts, 2003; Gorman & Sandefuhr, 2011). A different way of 


categorizing professions is to see them as the structural, occupational, and 
institutional arrangements for work associated with the uncertainties of 
modern lives in risk societies (cf. Evetts, 2013). Professionals are extensively 
engaged in dealing with risk, with risk assessment and, through the use of 
expert knowledge, enabling customers and clients to deal with uncertainty. 
To paraphrase and adapt a list by Olgiati and colleagues (Olgiati, Orzack, & 
Saks, 1998), professions are involved in birth, survival, physical and 
emotional health, dispute resolution and law-based social order, finance and 
credit information, educational attainment and socialization, physical 
constructs and the built environment, military engagement, peace-keeping 
and security, entertainment, the arts and leisure, and religion and our 


negotiations with the next world. 


Professionalism and Occupational Control 


A different version of this reinterpretation of the concept of professionalism 
has involved the use of Foucauldian concepts of legitimacy (Foucault, 1979) 
and of the control of autonomous subjects exercising appropriate conduct 


(Foucault, 1980). Analysis of legitimacy, as a property of both systems and 


actors, has been developed most fully in the work of Foucault and his 
followers on the nature of governmentality and the constitution of citizen- 
subjects within modern societies. In respect of professions as systems 
Foucault (1979) argued, following Weber, that the development of particular 
forms of expertise was a crucial element in the formation of governmentality 
from the sixteenth century onwards. This extension of the capacity to govern 
necessitated a shift in the basis of legitimacy. Acceptance of the divine right 
of the sovereign declined and was replaced by a discourse that held “popular 
obedience to the law” to be the sole source of legitimate rule (Foucault, 1979, 
p. 12). 

It is possible to use McClelland’s categorization (1990, p. 107) to 


differentiate between professionalization “from within” (that is, successful 
manipulation of the market by the group, such as in medicine and law) and 
“from above” (that is, domination of forces external to the group, such as in 
engineering and social work). In this interpretation, where the appeal to 
professionalism is made and used by the occupational group itself (“from 
within”), then the returns to the group (in terms of salary, status, and 
authority) can be substantial. In these cases, historically the group has been 
able to use the discourse in constructing its occupational identity, promoting 


its image with clients and customers, and bargaining with states to secure and 


maintain its (sometimes self-)regulatory responsibilities. In these instances 
the occupation is using the discourse partly in its own occupational and 
practitioner interests but sometimes also as a way of promoting and 
protecting the public interest (e.g. in medicine). 

In the case of organizational professionalism, as most contemporary 
public service occupations and professionals now practicing in organizations, 
professionalism is being constructed and imposed “from above” and for the 
most part this means by the employers and managers of the public service 
organizations in which these “professionals” work. Here the discourse (of 
dedicated service and autonomous decision making) is part of the appeal (or 
the ideology) of professionalism. These ideas of service and autonomy are 
what make professionalism attractive to aspiring occupational groups. When 
the discourse is constructed “from above,” then often it is imposed and it is a 
false or selective discourse because autonomy and occupational control of the 
work are not included. Rather, the discourse is used to promote and facilitate 
occupational change (rationalization) and as a disciplinary mechanism of 
autonomous subjects exercising appropriate conduct. Organizational 
professionalism is clearly of relevance to the forms of public management 
currently being developed in the United Kingdom, and more widely, in 
educational institutions (schools and universities) and in National Health 
Service hospitals and primary care practices. 

To conclude: the appeal to professionalism can and has been interpreted 
as a powerful motivating force of control “at a distance” (Fournier, 1999, 
following Miller & Rose, 1990). These interpretations can also assist in 
understanding the appeal of professionalism as a mechanism of occupational 
change in the modern world. It is also effective at the micro-level where 


essentially it is a form of inner-directed control or self-control where close 


managerial supervision is not required — professional workers do not need 
supervisors. Then organizational professionalism will be achieved through 
increased occupational training and the _ certification of the 
workers/employees, a process Collins (1979) labeled “credentialism.” 
Noordegraaf (2007) argues that in the public sector (e.g. healthcare, social 
work) we see a “hybridized” professionalism — that is, professional work that 
has already internalized the problematic of control, and deals as much with 
controlling work as with executing a service to clients. 

We now leave the sociological concern of occupational control and turn 
back to the discussion of professionalism as knowledge-based work. The 
following, final, subsection provides a bridge to psychology and discusses the 


epistemology of professional work. 


Epistemology of Professional Work 


Professions are epistemic communities, that is, groups or networks of experts 


who share knowledge and beliefs that are in general linked to values and 


interest (cf. Adler et al., 2008; Haas, 1992). From an epistemological point of 
view, professions represent classes of theories about our world, each 
profession developing its agreed terminology, a set of assumptions, and 
paradigmatic cases that exemplify how to apply knowledge in various 
practical scenarios. Although professions are epistemic communities, they 
have strong ties to practical work and its organization. Moreover, in 
professions as epistemic communities, knowledge is linked to social values 
and interests. The medical profession is tied to the value of health; the law 
professions to justice. Beyond this social service dimension, all professions 
strive for social recognition and influence. 

For studying the epistemology of professional works, several strands of 
research have examined how professionals apply or rework their knowledge 


in practical cases. For example: 


e Situated cognition, communities of practice, situated thinking (cf. 


Lave & Wenger, 1991): how work is defined and cognitively steered 


by specific interpersonal work settings. 


e Naturalistic decision making (see Chapter 25, this volume, by Mosier, 
Fischer, Hoffman, & Klein): how professionals decide in practical 


cases — beyond decision theory. 


e Epistemic cultures (Jensen, Lahn, & Nerland, 2012; Knorr Cetina, 


1999): how knowledge shapes professional work and the 


professionals’ identities. 


e Mental models (Johnson-Laird, 1983; Jones, Ross, Lynam, Perez, & 
Leitch, 2011): how professional work is defined by the execution and 


redefinition of professional mental models. 


A common finding is mediation (Engestr6m, 2007; Hakkarainen, Palonen, 


Paavola, & Lehtinen, 2004), that is, concrete professional work is mediated 


by tools, artifacts, machines, formulas — and today, in particular, by 
computers. For example, healthcare professionals not only request and 
interpret blood tests or radiographs, but are also able to conduct ultrasonic 
testing, heavily supported by computers. 

It is not easy to study the epistemology of professional work within the 
workplace, for several reasons. First, we must differentiate professional work 
from human work in general. For instance, the phenomenon of mediation has 
already been described by phenomenology (e.g. Heidegger; Bergson), as a 
merging of the actor and their instrument. Second, the work of linked or 
similar occupations might become indistinct, due to organizational 
constraints and procedural necessities. For instance, the person-processing 
conducted by a doctor or nurse might appear similar due to predefined steps 
in patient care. Third, the workplace is the common turf — not to say 
battlefield — not only for concurring occupations, but also between 
disciplines, such as the sociology of work, the psychology of work, and 
human resource management. 

When we follow Abbott’s (1988) epistemological analysis of 
professional work, abstraction also comes into play. According to him, 
professions have to be seen within a system. The system is centered on work, 


and consists of professions and their links to particular tasks. Abbott calls the 


link between a profession and its tasks “jurisdiction.” The professions 
compete with one another for control of particular tasks. The “currency” of 
this competition is knowledge (Abbott, 1988, p. 102). Abbott claims that 
interprofessional competition is based on a specific kind of knowledge, 
namely abstract knowledge: “For abstraction is the quality that sets 
interprofessional competition apart from competition in general” (pp. 8-9). 
To view professional work connected to abstraction is not unusual in the 
sociology of the professions. It is in line with Hughes, who speaks of 
“detachment” in professional work: “Having in a particular case no personal 
interest such as would influence one’s action or advice, while being deeply 
interested in all cases of the kind” (Hughes, 1965, p. 6). Similarly, Parsons 
speaks of the “intellectual component” of professional work (Parsons, 1968, 
p. 536), whereas Freidson refers to “formal knowledge” (Freidson, 1986, 
passim). 

According to Abbott, abstraction takes two forms — reduction and 
formalization. First, abstraction can be reduction in the sense of lack of 
content, i.e. “abstract which refers to many subjects interchangeably” 


(Abbott, 1988, p. 102). For instance, psychology claims that alcoholism is a 


personality disorder, thus falling into the domain of general psychotherapy. 
However, by abstraction, alcoholism may also be seen as a mere medical 
problem or a task for social workers. In any case, alcoholism is “reduced” to 
an aspect, from the perspective of an epistemic community. Second, 
abstraction is formalization, saying “that knowledge is abstract that elaborates 
its subjects in many layers of increasingly formal discourse” (Abbott, 1988, 
p. 102). Formalization means that a profession provides a formal system for 
interpreting particular types of problems. The language of modern medicine 


can be seen as one such formal system, providing a diagnostic system for 


tasks such as appendectomy. Formalization strengthens the jurisdiction of a 
profession. “No one tries to explain particle interactions without mastering 
the abstract knowledge of physics. More practically, no one offers insurance 
companies advice on underwriting without having mastered actuarial theory” 


(Abbott, 1988, p. 103). Both the difference between two forms of abstraction 


and their different linkages to inter- and intra-professional competition could 
be confirmed by studies on the market of environmental services (cf. Mieg, 
de Sombre, & Naef, 2013). 

When we tie the epistemological view on professions with that on 


workplaces, two questions arise that require further research: 


1. What is the interrelation of concrete working knowledge and the body 
of abstract knowledge of a profession? Hughes’s (1965) focus was on 
how abstract knowledge is cast into professional work and advice. 
Today, the focus is the other way around: how experience from concrete 
work plays into a profession’s body of knowledge. Of particular interest 
are epistemic “objects” (Knorr Cetina, 1999); that is, psychical objects — 
including people — that resist being entirely explained, and that 
potentially show aspects that are as yet undiscovered. As “boundary 


objects” (cf. Carlile, 2004), they stipulate interdisciplinary work. 


2. What is the role of formalization in promoting professional standards 
and support technologies ? Most of the standards and tools used by 
professionals have been developed by previous generations of 
professionals within that domain, and are now incorporated in 


technologies, standards, or even work structures. This phenomenon has 


also been addressed as “epistemification” (Jensen et al., 2012; Stutt & 


Motta, 1998). Today, formalization requires the cooperation of 


professions with other social institutions such as science, industry, and 


state bureaucracies (Mieg et al., 2013). 


Having discussed professional work and its epistemic foundation, we now 
turn to science as the core accepted reference system for knowledge in 
current societies. Whereas professionals are expected to apply scientifically 
based knowledge, scientists are responsible for developing and extending 
science as a knowledge base — while today also being professionals 


themselves. 


Science 


Science is professionalized but has its own history and forms of community. 
In this second part of our chapter, we start with a historical overview, 
introduce how science became institutionalized, and then describe operating 
conditions of today’s science, including disciplines and peer review. Finally, 
we examine the new co-production of knowledge, opening science for closer 


cooperation with society. 


Historical Overview 


Although the scientific method has its roots in antiquity, the scientist as a 
distinct role did not appear before the end of the seventeenth century (cf. 


Ben-David, 1965). Until that time, science was integrated within the roles of 


physician, teaching scholar, monk, or philosophical author. For instance, 
Copernicus (1473-1543) served as physician, administrator, and canon; 
Newton (1642-1726) was not only a fellow of Trinity College (Cambridge) 
but also Warden and then Master of the Royal Mint; and Leibniz 
(1646-1716) was installed as Privy Counsellor of Justice and Librarian of the 
ducal library. The term “scientist” first appeared, in English and German, in 
encyclopedias and dictionaries of the nineteenth century (Ross, 1962). 

The history of science is regarded as having started with the Greek 
philosophers. Early scholars such as Thales (sixth century Bc) or Euclid 
(third century pc) formulated ideas and principles within natural philosophy 
that are still relevant to modern science. Pythagoras (sixth century Bc) and 
Archimedes (third century sc) contributed to the fundamentals of 
mathematics. Aristotle (384-322 Bc) integrated Greek philosophy into a 
systematized view of the world and defined the scientific methods. The 
Encyclopedia Britannica declared Aristotle as “the first genuine scientist in 
history” (Gribbin, 2008, p. 12). This is true with regard to his scientific 
worldview; in his professional life, he represented the classical Greek savant: 
he was consultant, writer, and teacher of Alexander the Great. Guthrie even 
called the Greek sophists — a famous group of consulting philosophers — a 
“profession.” “They were itinerant teachers, who made a living from the new 


hunger for guidance in practical affairs, which arose at this time [second half 


of the fifth century pc]” (Guthrie, 2004, p. 66). The rise of science within 


Greek philosophy happened in an era of 
e relative wealth and expansion by maritime trade; 


e friendly competition between self-regulated colonies (sports, music, 
theatre plays) with opportunities for public performance (agora, 


theatre); 
e aculture of the written word. 


Greek philosophy would have been lost in the turmoils of the Migration 
Period — the Dark Centuries — without the scholars of the Islamic world and 
the Christian monasteries (cf. Lindberg & Shank, 2013). The Greek tradition, 
in particular Aristotle, became the central momentum of the scholastic 
tradition, most often serving theology and dealing with metaphysics or 
theodicy. One of the greatest scholastics, who also contributed to advancing 
the scientific method, was William of Ockham (c.1287—1347). He defined a 
core principle of ontology and epistemology (“Occam’s Razor,” of several 
possible versions): “Entia non sunt multiplicanda praeter necessitatem” — new 
entities should not be introduced without good reason. 

A new era in the formation of science started with the Italian 
Renaissance, in an environment of rich Italian city-states competing with one 
another for maritime trade. Here we see the emergence of the artist—scientist, 
a “profession, that of the artists, for whom philosophy was first and foremost 
science” (Ben-David, 1965, p. 27). The artist—scientist was considered a 


“virtuoso” (Ben-David, 1965, p. 37), like Galileo Galilei: “a man who could 


be consulted on great engineering and architectural projects and show his 


brilliance in other serious and playful ways, he was honoured as a man of 


outstanding imaginative talents” (Ben-David, 1965, p. 37). However: “The 
hybrid identity of artist—-scientist was a transient phenomenon” (Ben-David, 
1965, p. 29). 

The consolidation of the scientific role in the seventeenth century went 


along with a “protestant science policy” (Ben-David, 1965, p. 44) in Northern 


Europe. Protestants were “disproportionately highly represented among 
scientists from the sixteenth to the end of the eighteenth century” (Ben- 
David, 1965, p. 46). This correlation was first observed by Merton (cf. 
Merton, 1973b) and has since been discussed as possibly a phenomenon of 


culture or of class (cf. also Becker, 2011). Ben-David (1965) resumed: 


“Scientific activity was welcomed by those who were interested in a more 
secular education and who shared the distaste of anything that reminded them 
of the old regime” (p. 44). 

In that time, science became linked to the idea of progress. The key 
figure was Sir Francis Bacon (1561-1626), who defined the experimental 
method. The core idea was not simply to observe nature (as preferred by 
Aristotle) but to experiment with nature in a controlled manner. The 
following centuries were termed by Sagasti (2000) the “Baconian era”: the 
identity of science with progress. “Three key features distinguished this 
program from other views on the production and use of knowledge current in 
Bacon’s time: (i) an awareness of the importance of appropriate research 
procedures (the scientific method); (ii) a clear vision of the purpose of the 
scientific enterprise (improving the human condition); and (iii) a practical 
understanding of the arrangements necessary to put the program in practice 


(scientific institutions and state support)” (Sagasti, 2000, p. 596). 


Institutionalization 


Science functions within communities of people who share their scientific 


interest (cf. Moscovici, 1993). For this community, a critical mass is needed; 


“experts, particularly experts in science, are heavily dependent on other 
people’s knowledge, whether it is taken from textbooks or journals, taught, 
learned by imitation and coaching, or built into instruments” (Turner, 2014, 
p. 8). The scientific role is developed within and in the service of the 
scientific community. Ben-David (1965) claimed that, had a group of persons 
existed “who regarded themselves as scientists, anywhere in the Christian or 
the Moslem world or among the Jews,” then it would not have taken 
approximately a thousand years for the resumption of scientific inquiry, 
based on the earlier Greek achievements (Ben-David, 1965, p. 15). In the 
following we provide an overview on how, in the course of time, science in 
the Western world has been institutionalized with the help of: (i) universities, 
(ii) academies, and (iii) scientific journals. 

Universities. Universities are corporate communities of teachers and 
students (“universitas magistrorum et scholarium”). As students and teachers 
may originate from elsewhere — that is, from different legal systems — 
universities constituted a more or less autonomous, legally protected space 
for their members. “Universitas” defined the legal status of an autonomous 
guild (Shank, 2013). The first universities were established in Bologna 
(1088), Oxford (1096), and Paris (1200). With the universities “the 
specialised role of the university teacher emerged” (Ben-David, 1965, p. 19). 


However, despite the fact that the major contributions to science during the 


early times of the universities were made by those with a scholarly education, 


their discoveries were made outside universities. “Science was a marginal 
activity within the university and those who learned it there saw their 
knowledge of it as a peripheral feature in the image they had of themselves” 


(Ben-David, 1965, p. 26). The modern research university did not emerge 


until the nineteenth century. 

Academies. Academies are more or less independent institutions for 
scientific discussion and organizing research. In Europe, academies evolved 
in the context of royal or princely courts (Moran, 2006). For instance, the 
astronomer Johannes Kepler worked in the imperial court of Rudolf, in 
Prague. The model, as well as the name for academies originated in Plato’s 
Academia, a teaching institution for philosopher-noblemen north of Athens, 
which existed (despite interruptions) until 529 ap. The first modern academy 
was the Accademia dei Lincei in Rome, founded by the young Marchese di 
Monticello in 1603, followed by the German Leopoldina (1652), the Royal 
Society in London (1660), and the Académie des Sciences in Paris (1666). 
There was a preceding, parallel institution in the Islamic world, the Bayt al- 
Hikma (“House of Wisdom”) in Baghdad (until the siege of Baghdad in 
1258), where texts from Greek antiquity were collected and commented on. 

Journals. The main form of intra-science communication is through 
academic journals, which include empirical studies, research overviews, as 
well as theorizing. The oldest extant journal is the French Journal des savants 
(originally: “Journal des scavans”), founded in 1665 and renamed after the 
French Revolution. As the individual issues of an academic journal may have 
contained only a few pages, the issues were bundled in the form of volumes. 
As the page numbers refer to a continuous volume, only the first issue of each 
volume starts with page number one. As an example of certain long- 


established practices in academic reporting, Spier (2002) hints that writing 


reports for colleagues is an old tradition in medicine, already practiced by 


physicians of the Islamic Golden Age. 


Today’s Operating Conditions of Science 


Present-day science is professionalized; that is, people are paid for advancing 
science. However, science does not look like a normal profession with regard 
to the professional—client relationship. People have clear motivations for 
consulting a doctor or lawyer — they are sick or seek legal assistance. But 
who consults a scientist, and for which reasons? From a political perspective, 
the kind of service that science provides has to be questioned. Turner (2014, 
p. 1) notes: “Why would patrons, especially democratic governments, pay for 
something, namely science, that they didn’t understand and could not easily 
assess the value of?” One answer might be that science not only is a 
profession, but constitutes the core epistemic social (sub-)system within 
modern societies, responsible for methodologically legitimated production of 
“truth.” The following sections introduce the perspective of science as a 
social system and also as a profession, along with some core “operating 
conditions” of science, namely disciplines and peer review. 

Science as a social system. Modern societies are functionally 
differentiated. This means, for example, that as members of a modern society 
we are not required to assume personal responsibility for growing food nor 
for teaching our children how to write. The division of labor in our societies 
has developed social (sub-)systems that interact with one another, but every 
(sub-)system has its distinct logic, based on which it operates. According to 


Parsons (1951) and Luhmann (1995), we can distinguish, for instance, law as 


a distinct social system with its own institutions (e.g. courts) and 
occupational roles (e.g. judges, lawyers). In a similar way, science can be 


understood as a social system of its own, its function being the production of 


knowledge and striving for “truth” (as fallible and revisable as that truth 
might be). Stichweh (1992) described the evolution of science as a social 
system operating and reproducing itself along five “stabilizing factors”: 
occupational roles; the “school—college—university sequence”; universities; 
disciplines; and “disciplinary identity” as the professional identity of 
scientists (pp. 13-14). 

Science as a profession. Science considered as a profession has to be 
granted a certain autonomy, i.e. the right of self-regulation (Ben-David, 
1972). The other side of this coin is internal quality control, for instance 
through setting methodological standards or awards such as the Nobel Prize. 
More comprehensive, however, is quality control through internalized values. 
Max Weber and Robert K. Merton, discussing the role of scientists, pointed 
to the shared values of scientists. Weber, in his famous lecture on “science as 
a vocation” (1917), called for objectivity, that is, scientists should refrain 
from basing science on political worldviews. Merton (1973a) claimed that 
there are four sets of “institutional imperatives” guiding the work of 
scientists: universalism, communism, disinterestedness, and organized 
skepticism (p. 118). More recent studies describe scientific cultures (e.g. 
Knorr Cetina, 1981; Latour & Woolgar, 1986); that is, the attitudes and 
practices that professional science requires. Due to the pressure to produce 
significant scientific results in order to survive as a professional scientist, 
scientific misconduct has become an enduring phenomenon (cf. Casadevall & 
Fang, 2012). 

Disciplines. The main operating condition of professional scientific 
work continues to be defined according to disciplines (Mieg & Frischknecht, 


2014), which evolved in the eighteenth century (Stichweh, 1992), originating 


from a transformation of the ancient “liberal arts” in philosophy such as 


geometry or astronomy (cf. Cadden, 2013). Disciplines are the form of the 


institutionalized scientific community, the core epistemic communities (cf. 
Haas, 1992). They define and organize standards for specific work and 
scientific education, thus defining career paths and professional scientific 
identities. More importantly, disciplines channel funding for research (Ben- 
David, 1972), which Merton and Zuckerman termed a “gatekeeper role” 
(1973, pp. 522-523). Disciplines, as organized communities, may also 
introduce inequalities and power differences, including gendering, into 
science (Zuckerman, Cole, & Bruer, 1991). 

Peer review. Along with the scientific community, peer review has 
evolved as the main quality control mechanism in science. According to 
Chubin and Hackett (1990, p. 19), peer review was established with the 
Philosophical Transactions of the Royal Society in 1665. In the context of 
climate change research and politics, the transparency and reliability of the 
peer review process have become a widely discussed issue (Edwards & 


Schneider, 2001). Typically, a scientist who wishes to communicate their 


research findings to the community writes a paper and submits it to a journal. 
The journal’s editor sends it to two or more experts (the “peers”) who review 
the paper, make comments, and suggest whether the paper should be 
published in its present form, revised prior to publication, or rejected 
outright. Peer review is a “faute de mieux” mechanism — in the absence of 
better instruments. Many studies have shown the low reliability of peer 
reviews (e.g. Marsh & Ball, 1989). Edwards and Schneider (2001) warn 
against understanding peer review as a “truth machine” (p. 231) and 


underline its regulative function within the scientific community. 


The New Co-Production of Knowledge 


Since the 1990s, a new paradigm of science—society cooperation has evolved 
(Berkes, 1999; Gibbons et al., 1994; Jasanoff, 2004; Nowotny, Scott, & 
Gibbons, 2001; Thompson Klein, 2014). The underlining premise is: many 


urgent problems in our societies, ranging from environmental issues to 


transnational migration, are too complex to be understood by one scientific 


discipline or by “normal” science. According to a typology by Callon (1999), 
science—society cooperation follows the model of: (a) Speak to the public 
(astronomy, molecular biology); or (b) Debate with the public (e.g. genetics, 
nuclear energy). In contrast, the co-production of knowledge strives for 
“actively involving lay people in the creation of knowledge concerning them” 


(Callon 1999, p. 89): “In the third model the dynamics of knowledge is the 


result of a constantly renewed tension between the production of standardised 
and universal knowledge on one hand, and the production of knowledge that 
takes into account the complexity of singular local situations, on the other 
hand” (Callon, 1999, p. 89). 


This new concept of co-production of knowledge runs under several 


titles, for instance “mode 2” (Gibbons et al., 1994), “transdisciplinarity” 


(Thompson Klein, 2014), or “responsible research and innovation” (von 
Schomberg, 2013). The core idea of this new form of science—society 
cooperation is that socially relevant research tasks have to be jointly defined 
by scientists and the non-academic “owners” of the problems to be tackled, 


resulting in “mutual learning” (Scholz, Mieg, & Oswald, 2000). This view 


involves a redefinition of expert roles (cf. Edelenbos, van Buuren, & van 


Schie, 2011; Mieg, 2006; Nowotny et al., 2001). In addition to scientists, 


science—-society cooperation involves politicians, other professionals, 
stakeholders, residents, and public administrations. There is a particular focus 


on the local knowledge of residents or stakeholders (cf. Carolan, 2006); Mieg 


(2006) speaks of “system experts,” as residents may have gained more or less 
explicit knowledge about a local human—environmental system through their 
long-term experience. This kind of system knowledge differs according to the 
personal or professional activity and commitment to a place. A set of 
methods has evolved for the integration of laypeople or external expertise 
into transdisciplinary processes (e.g. Perera, Drew, & Johnson, 2012; Scholz 
& Tietje, 2002). 

The changing relationship between science and society makes it 
necessary to rethink the expert roles involved, particularly that of local 
expertise or “system experts.” Having discussed professionals in general and 
scientists as specific professionals, we now turn to an integrated view of what 


it is to be an expert. 


Expert Roles 


Scientists and professionals are perceived to be experts by the general public. 
In this final part of our chapter, we reflect on the social roles of experts in 
general terms. We start with a view of experts as an exclusive group, forming 
a powerful elite, while also seeing experts in a counter-position of 
undermining political power. There follows a contrasting (non-elitist) view of 
the expert role as a general social form — with ways of implementation into 
the social division of labor. We conclude by reflection on the question, Why 
do we need experts, and in what context? We propose a recontextualization 
of expertise, including professionalism and excellence as two different forms 


of expertise. 


From Elitism to Interactional Expertise 


Scientists and professional experts might be considered as a social elite. 
There is no standard definition of “elite” in social science. However, current 
definitions generally share three core features: Elites are small groupings of 
persons with (i) a high degree of potential power, due to (ii) formal positions 
or “charisma” of a person, based on (iii) a process of selection (cf. Carlton, 
1996). The notion of “elite” was introduced within Italian sociology, as an 
alternative concept to Marxist egalitarian concepts. With reference to the 
ancient idea of aristocracy (aptotoc = the best), Pareto (1963) defined elites 
as those who are most capable in any area of activity. In present-day 
definitions of “elite,” the emphasis is placed on power, not on capability 


(Etzioni-Halevy, 1993). The power of elites is based on the possession and/or 


control of various resources. 

In current societies, there may sometimes be a conflict between 
democratic control and rational administration through experts. As expertise 
is not easily comprehensible by lay citizens, democratic control of the 


expanding activities of experts is more questionable than ever (Etzioni- 


Halevy, 1993; Feyerabend, 1978; Turner, 2014). In order to maintain the 
capability to meet public demands and maintain an equilibrium between 
public and particular interests, elites must admit selected members of the non- 
elite and exclude those who might damage their reputation through 
incompetence, violation of public morality, or excessive parasitism on public 
goods. This process of self-purification is called “circulation of elites” 
(Kolabinska, 1912) and is particularly characteristic of American elites 
(Lemer, Nagai, & Rothman, 1996). 


The circulation of elites is also driven by the emergence of counter- 
elites (cf. Rucht, 1990): current societies have to cope with unintended, 
undesirable side effects of human activities, and with newly emerged natural 
threats. In many cases, individual experts or small groups of experts (often 
scientists) first anticipate or perceive such a problem, conduct research on the 
issue, and attempt to initiate public discussion (environmental issues being 
contemporary examples). During periods of controversy, apologist experts 
constitute a counter-elite to established elites that remain reluctant to 
recognize the issue as a problem or the solutions recommended. If the issue 
as a matter of public concern becomes indisputable and its solutions 
standardized, the counter-elite becomes a newly established elite, and parts of 
former elites may be forced either to resign or to join in. 

The elite view of expertise might appear plausible for scientists and 
other professionals; however, it leaves unappreciated a form of experts that, 
according to Collins and Evans (2007), play a considerable role in modern 
societies: interactional experts (see Chapter 2, this volume, by Collins & 
Evans). Interactional experts are able to intermediate between specialized 
domains and other fields of application. Collins and Evans (2007) define 
interactional expertise via domain-specific language competence: “This is 
expertise in the language of a specialism in the absence of expertise in its 
practice” (p. 28). Specialist journalists, managers, coaches, and salespersons 


are examples of interactional experts (Collins & Evans, 2007, pp. 30-32). For 


instance, diet coaches interpret and apply knowledge from the health sciences 
to their clients. Interactional expertise “provides a bridge between the rest of 
us and full-blown physically engaged experts, and it touches on a wide range 


of professional activities” (Collins & Evans, 2007, p. 77). 


Expert Roles and the Social Division of Labor 


What is the core social mechanism within expert roles? The answer seems 
linked to how we use other people’s expertise. Let us consider the following 


situations: 
e asking someone on a street for directions to the rail station; 
e consulting a doctor about one’s acute health problems; 
e speaking in court as an appointed expert on asbestos; 
e instructing a child how to lace their shoes; 


e advising farmers on how they should prepare to cope with extreme 


weather conditions. 


These situations differ in various aspects: the persons and consequences 
involved; the frequency and probability of the event; and its general social 
significance. Even the perspectives differ. In some of the situations, we are 
asking somebody; in others, we are answering in some way. Irrespective of 
how the situations may differ, they share the same form. Somebody explains 
an issue (what, how, and/or why) to someone else. In the following, this 
social form is called “The expert” or, simply, the expert role. 

The power of the social form “The expert” can be seen clearly in 
unstructured contexts such as a group of people who are unfamiliar with one 
another and meet only once. This is the situation we encounter in temporary 
groups. Garold Stasser and colleagues have studied the effects of expert role 
assignment in experimental groups. A common experimental design is the 


hidden profile: Some group members have unshared information that is 


necessary to complete the overall group task (cf. Stasser & Titus, 2003). In 
these studies, unshared information is a basis for the definition of expertise, 


signifying “that a person has access to more information in a specific domain 


than others in the group” (Stewart & Stasser, 1995, p. 619). It can be shown 
that the explicit assignment of particular expert roles to the group members 
who have unshared information increases the chance that this piece of 
information will contribute to the group’s work. From this and other results 
on expert role assignment, Stasser and colleagues inferred conditions for 
effective expert role assignment in groups (see Stasser, Stewart, & 
Wittenbaum, 1995, p. 263): 


e the members’ expertise status needs “to be explicitly recognized 


within the group”; 


e members need to be “mutually aware of each other’s area of expertise 
at the onset of discussion to facilitate dissemination of unshared 


information”; 


e expert role assignments need to fit the actual distribution of 


knowledge; 


e there should be no competition (cf. Toma, Vasiljevic, Oberlé, & 
Butera, 2013). 


We can generalize the experimental evidence to understand expert roles as a 
social form of interaction (Mieg, 2001). Figure 9.1 depicts the constituents of 
“The expert” interaction: a person is addressed as an expert in front of a “lay” 
audience (a client, layperson, jury, TV audience, etc.). The person is 
addressed because he or she might have knowledge relevant to tackle a 


certain problem. The kinds of problems vary slightly with the context: the 


patient consults a doctor, seeking pain relief; a jury is comprehensively 
informed by an expert on the health risks associated with asbestos in a 
particular industrial setting; or a stranger simply requires directions to the rail 
station. There are always some questions associated with the attribution of 
“expertise.” Does the “expert” really have the specific knowledge required 
(expertise)? Is this kind of knowledge really appropriate to tackling the 


identified problem (relevance)? 
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Figure 9.1 The constituents of the expert role. 


In general, the expert role is embedded within other roles. The pure form 
of “The expert” can be found in courts (cf. Mieg, 2001, pp. 49-55): experts at 
court are witnesses, testifying knowledge. Most often, the expert role is 
attributed to professionals such as scientists or doctors. For professions, the 
expert role is connected to occupational interests with direct impacts on 
personal income and status. In the context of co-production of knowledge 
(see above), system experts such as farmers or people with local expertise are 
consulted. In many ephemeral cases, we make use of relative experts — 
people whom we trust to know more about a certain aspect of life, for 


instance when asking someone on the street for directions to the station. The 


relative expert status can shift depending on the domain discussed (Jacoby & 
Gonzales, 1991). 

The expert role is a core mechanism in the social division of labor. By 
experts, we make use of expertise incorporated in other people — the experts. 
As Mieg (2001) put it: the “expert” is a social form through which someone 
is attributed as “expert” for time-efficient use of knowledge. We find this 
social use of expertise in different social contexts such as groups, companies, 
as well as society as a whole. Research on expert roles in the context of the 


division of labor encompasses: 


e Research on the distribution and coordination of tasks and 
competencies in groups such as couples, families, or organizations, 
forming a “transactive memory” (cf. Peltokorpi, 2008; Wegner, 
1987). For specific information, we refer to a specific group member 
who is responsible for storing and processing this sort of information 


(birthday dates of relatives; new CAD programs, etc.). 


e Research on intermediate social structures such as professional 
associations and scientific communities or inter-professional expert 
networks; as Edwards (2011) argued, inter-professional cooperation 
requires “relational expertise” and “relational agency,” that is, 
Capacities to understand others’ expertise and organize distributed 


work. 


e Research on gatekeeping. A decisive “interactional” role is played by 
“gatekeepers” in companies; they are receptors for technological 
trends and translate them into product options (OECD, 2005; Rau & 
Haerem, 2010). 


e Research on networked expertise, representing “higher-level cognitive 
competencies” (Hakkarainen et al., 2004, p. 9), ranging from informal 
collaboration within a company or local and regional clusters (cf. 
Clark, Gertler, Feldman, & Williams, 2003) to more formalized 


cooperation within national innovation systems (OECD, 1999). 


e Research on institutionalized expertise in the forms of companies, 
universities, public administrations, or “centers of excellence” (Mieg, 
2014). 


From informal teams to national administrations, expert roles help to make 
use of available, distributed knowledge. Hereby, the social form “The expert” 
triggers trust due to its in-built reversible and potentially inverse character: if 
I had enough time to commit myself to a specific domain of knowledge and 
skills, I could contribute as an expert to the social division of labor. 

Let us turn finally to approaches that attempt to integrate research on 


professionalism, scientific expertise, and expert roles. 


Two Dimensions of Expertise: Excellence and Professionalism 


Let us consider expert work (within the division of labor) as an activation of 


expertise in context (cf. Hoffman, Feltovich, & Ford, 1997). Studying 


expertise, we face two dichotomies that are linked to domain differences: 


1. Expertise as the base of extraordinary performance vs. expertise as the 
base of an ordinary occupation: the psychology of expertise tends to 
view experts as the outstanding performers of a domain (cf. Ericsson, 
2006). This makes sense for domains such as chess or sports, but is less 


obvious in domains such as health or social work. 


2. Competence as valid judgment vs. competence as effective help for a 
person in need. From a scientific point of view, the work of experts 
should consist of or be based on accurate judgments (cf. Weiss & 
Shanteau, 2014); however, valid scientific judgments seem to be out of 


scope for domains such as social work — here, effective help is needed. 


There are few comparative studies on domain effects within the psychology 
of expertise. In the 1990s, Shanteau (1992) argued that there are separate 
performance classes of expert judgments depending on the domain, with 
performance being measured by the reliability and validity of judgment. We 
find consistently high accuracy of judgment by experts in domains such as 
weather forecasting, chess, or soil quality, and low judgment accuracy by 
court judges, clinical psychologists, or stockbrokers. The findings for 
physicians diverged. For example, radiologists show good decision 
performance, psychiatrists a poor one. Shanteau (1992) presented several 


explanations for this dichotomy. On the one hand we have domains that 


involve decisions about natural objects with stable properties and predictable 
stimuli; on the other hand we have to make judgments on human behavior 
with less predictable stimuli. Another explanation might be the difference in 
learning from feedback. In an update, Shanteau (2015) adds that high- 
performance judgments are often aided by computerized tools such as in 
weather forecasts. 

A solution to the domain puzzle might be to reconceptualize expertise. 
Mieg (2009) argues that professional expertise (beyond relative expertise) 
consists of two related dimensions — one clearly defined by individual 
excellence, the other related to professional engagement, a dimension that can 
be called professionalism. Excellence heavily depends on deliberate practice, 
and to a lesser extent on education. Excellence is the kind of expertise that is 
measurable by standardized or “representative” tasks, as claimed by Ericsson 
(2006). The second expertise dimension is “professionalism.” This depends 
on socio-cognitive competence (e.g. communicative and organizational 
skills), and is correlated with the commitment to a_ profession. 
Professionalism is needed to turn excellence into paid work. 

Figure 9.2 depicts the two dimensions and their correlates, and 
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tentatively adds four additional factors: “talent,” “family,” “science,” and 
“society.” Talent appears to be a prerequisite in many domains such as music 


or acting (for a discussion, see Ericsson, 2014; Hambrick et al., 2014). The 


family background is influential, as discussed in the context of elites: elite 
families of doctors, lawyers, entrepreneurs, or politicians provide the cultural 
capital and necessary resources for the careers of their offspring. “Science” 
refers to technologies and systematized knowledge that help both to develop 
expertise and to execute expert work. Finally, “society” encompasses cultural 


and institutional provisions such as national educational systems or 


regulatory frameworks that support the organization and execution of expert 


work in a society. 
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Figure 9.2 The factors of expertise. 


Conclusion 


To conclude, the study of professionalized expertise requires an integrated 
approach (cf. Gobet, 2015). We need to better understand why, in the words 
of Shanteau (1988), experts are “essential in precisely those domains where 
there are no right answers” (p. 206). Shanteau adds: “Even when standards do 
exist, as in auditing, it is experts who establish these standards and who have 
the power to change them. Thus experts define the standards, not the other 
way around” (p. 206). Professionalism is needed to establish standards in a 
domain — including “representative tasks,” that is, standards to enforce the 
professionalization of a domain and to render possible, to identify, and to 
foster excellence within this domain. As Mieg (2009) shows with respect to 
professional environmental services, professionalism dominates in those 
fields where new standards for best practice need to be established. 

At the outset of this chapter, we stated two historical trends, first an 
ongoing differentiation and reworking of domains of expertise, and second 
the (self-)organization of experts, e.g. by developing quality standards and 
training facilities. For the study of expert work, at least two puzzles remain to 


be solved: 


1. The puzzle of domains. How can we understand domain differences 
in expert performance? What can we learn from a historical account of 
domains, e.g. the history of sport, which shows how performance 
increased due to improved training methods and biophysical knowledge 


(cf. Ericsson, 1996, 2006)? Or do we observe objective constraints that 


limit expert performance, such as in forecasting financial markets — 


tasks that have nevertheless become professionalized (cf. Mieg, 2001)? 


2. The puzzle of institutionalization and/or innovation. How might 
the institutionalization of expert work, e.g. in the form of professions or 
science, not only foster expert performance but also inhibit innovation? 
If innovation arises at the boundaries of knowledge systems, can there 
be any systematically trained expertise for innovation? How shall we 
understand the evidence for both common knowledge (Carlile, 2004) as 


well as formal knowledge (Mieg et al., 2013) in advancing innovation 


that arises from expert work within our societies? 
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Introduction 


It is a truism that perception changes across development. Visual acuity 
improves over the first several months of life (Dobson & Teller, 1978), and 
then slowly decreases through aging. Older viewers have quite different 
patterns of acuity across lighting levels and noisy scenes than do younger 
viewers (Haegerstrom-Portnoy, Schneck, & Brabyn, 1999); in general, 
sensory acuity changes drastically across the lifespan. Furthermore, it is clear 
that early experiences affect later perception, even at very long time delays. A 
classic example of such a “sleeper” effect is that children born with extensive 
bilateral cataracts removed through surgery when less than 6 months old 
appear to regain their sight perfectly, but fail to form high-level integrated 
categories (e.g. faces) normally several years later (Le Grand, Mondloch, 
Maurer, & Brent, 2001). Because basic perceptual mechanisms have a huge 
impact on what we are able to do, and play a critical role in the development 
of expertise, the development of perception is intertwined with the 
development of expertise. This chapter will review three interrelated strands 
of research into the relationship between general expertise and perceptual 
processes: at the neural level, at the level of basic processes, and at the level 
of complex coordinated behaviors. 

Since at least Bruner’s classic “new look” movement, researchers have 
been wondering how, if at all, personal experience affects the nature of 
perceptual experience (Bruner, 1992; Miller, Bruner, & Postman, 1954). 
Bruner argued that perception was directly and predictably influenced by 


experiences, reporting that, for instance, poor school children with little 


experience of money estimated the sizes of viewed coins as larger (Bruner & 
Goodman, 1947). Though recent work has questioned these specific results 
(Dorfman & Zajonc, 1963; Klein, Schlesinger, & Meister, 1951), the basic 


question of whether “pure” perceptual experience itself is somehow altered in 


the course of the acquisition of expertise remains open and fascinating. One 
specific way to think about this general kind of perceptual expertise is in 
terms of categories: perceptual experts are ones with massive experience with 
a particular category of objects (in the literature, often birds, dogs, and cars), 
who can make fine discriminations over these visual categories. 

This chapter will emphasize perceptual experience and its role in 
expertise. Perception is essentially never really separate from action (e.g. 
Glenberg, 2010), particularly in the deployment of expertise, since expert 
behaviors are essentially always at some level directed toward a target, by 
definition. Moreover, many perspectives on the interaction between skill and 
perception, including most discussed here, implicitly or explicitly hold that 
perception and action loops form an important substructure binding together 
world events and skills. Nonetheless, to the degree that it is convenient and 
possible, this chapter will emphasize the relationship between expertise and 
perception, invoking interconnections with motor behaviors only when they 
are most directly relevant. 

This review will explore traditional questions of how perceptual 
experience is affected by experience. Our emphasis, however, will be on a 
distinct question that has received increasing attention in recent years: how 
does expertise guide the deployment of perception in complex tasks? Over 
the last 30 years and more, a wealth of research suggests that learning in 
complex areas crucially involves learning to see well: perception can be 


tuned to highlight task-relevant features and deep properties (Chase & Simon, 


1973; Chi, Feltovich, & Glaser, 1981; Hegarty, Keehner, Khooshabeh, & 


Montello, 2009), to attend to important aspects of problems to control the 


time course of reasoning (Gleicher, Correll, Nothelfer, & Franconeri, 2013; 
Salvi, Bricolo, Franconeri, Kounios, & Beeman, 2015), to recruit and 
restructure neural regions to accomplish skilled perceptual tasks (Gauthier, 
Skudlarski, Gore, & Anderson, 2000; Gauthier, Tarr, Anderson, Skudlarski, 


& Gore, 1999), or even to perform important conceptual tasks which are 


offloaded onto perceptual-motor processing networks (Clark, 1998; Landy, 
Allen, & Zednik, 2014; Zhang & Norman, 1994). Part of expertise may lie in 


learning appropriate perceptual skills. For example, geoscientists, who must 


work with rocks that have been fractured by past geological episodes, show 
an improved ability to recognize word-forms that have been fractured in ways 
rocks typically are (Newcombe & Shipley, 2015; Shipley, Manduca, Resnick, 
& Schilling, 2009). 


The question “does experience and skill alter pure perceptual 


processes?” is a powerful one (for comprehensive reviews of the evidence on 
this question, see Firestone & Scholl, 2015), but is quite specific; such a 
dichotomous question can limit what we are prepared to view as interesting, 
and in the limit can bog progress down in unproductive methodological 
conflicts, or mask unclear operationalizations of the core question — which is 
after all somewhat intrinsically vague. Which perceptual processes count as 
“pure”? Do alterations in the role the process plays count, or only changes in 
its internal structure or phenomenology? Similarly for the traditional 
Whorfian formulation of whether language (or experience) affects “thought.” 
Neither thought nor pure perception are well enough conceptualized to expect 
interesting results from such vague questions. 


The focus in this chapter is instead on the many different ways that 


perception and perceptual-motor activity play a role in the acquisition and 
deployment of expertise; we hope that this question provides a broad context 
to explore potential interactions between learning and perception. We divide 
the discussion into three sections, addressing roughly three separate 
questions: (1) What role does the acquisition of complex perceptual routines 
play in high-level activity? (2) How do perceptual regions of the brain (and 
perceptual processing) organize to support expertise? (3) How does expertise 


affect basic perceptual behaviors? 


The Role of Perceptual Routines in Expertise 


While the idea that a core set of perceptual mechanisms remains largely 
unaltered by experience remains perhaps viable, it is clear that the 
deployment of those mechanisms is massively altered by experience and 
skill. Over thirty years ago, Ullman (1984) suggested that visual object 
recognition worked not just via statically deployed representation-building 
processes, but through temporally extended “routines” — shifts of attention, 
mental rotations and alignments, and other basic operations — the deployment 
of which in correct sequences implemented visual tasks. This perspective 
assumes a mechanism by which perception could be overtly strategic; and 
indeed, it has been found that in some cases of complex perception (e.g. 
sexing chickens), adopting overt attentional strategies can lead to rapid 
improvements in visual discrimination (Biederman & Shiffrar, 1987). 

At the same time, researchers in high-level cognition have long 
recognized the centrality of appropriately tuned perception — perceptual 
processes that see principally the important properties of a display or scene 
(Chase & Simon, 1973; Chi et al., 1981). Putting together the idea of high- 
level perception in complex tasks and the notion of visual operations suggests 
that recognition of complex situations and structures relies on an acquired set 
of visual operations that align mental models and physical displays. For 
instance, dentists examining teeth might learn structured patterns of focal 
locations and visual features, which align with a well-developed mental 
model of tooth structure and damage patterns (Hegarty et al., 2009). If this 
kind of account is right, then the deployment of expert knowledge should be 


impacted by visual features such as salience and grouping that impact the 
deployment of perception — a prediction confirmed in the cases of weather 
map comprehension (Hegarty, Canham, & Fabrikant, 2010) and arithmetic 
calculation (Braithwaite, Goldstone, van der Maas, & Landy, 2016). In both 
cases, novices are largely unaffected by visual salience, but those with more 
knowledge and expertise are more strongly affected by the visual features of 
expertise-relevant displays. 

Acquired perceptual strategies seem to be a part of many different high- 
performance fields one might have thought, on the surface, to be conceptual 
in nature. Becoming an expert meteorologist involves learning a large 
number of facts about weather. But it also involves learning how to look at a 
weather map so as to efficiently extract information from it (Hegarty et al., 
2010). Expert chemists acquire mental transformation strategies suited to 


particular representations (Stull, Hegarty, Dixon, & Stieff, 2012), and expert 


geologists seem to acquire mental transformations particularly suited to 
tracing temporally ordered shearing and deformation events — the kinds of 
events that shape the patterns of geological events commonly understood by 


geologists (Newcombe & Shipley, 2015). Each of these cases illustrates that 


part of expertise is learning to select appropriate features and visual 
transformations; the typical theoretical assumption is that what is learned is 
not basic perceptual mechanisms, but more how sequential operations of 
those mechanisms can be efficiently deployed in the service of complex 
reasoning. For instance, children learn to read graphs by aligning verbally 
stated relationships (e.g. “more blueberries than oranges”) with particular 
looking patterns, e.g. left-to-right ones (Michal, Uttal, Shah, & Franconeri, 
2016). 


In other cases, perceptual strategies seem to involve attending to 


complexes of features different from the immediately available object. 
Franconeri and colleagues, for instance, have demonstrated that expert 
reading of bar graphs for interactions and main effects involves the 
introduction of new perceptual features -— in particular ensemble 
characteristics. To compare the values across one condition of a 2x2 
interaction, many experts attend to and compare the mean of each pair of 
bars. That is, new locations must be projected based on other perceived 
values, and these imaged values must then be sequentially attended (Szafir, 
Haroz, Gleicher, & Franconeri, 2016). Eye-tracking work suggests that 
tracking linear functions, in some contexts, involves the projection of 
properties or attentional markers into scenes, which then serve as the subject 
of focused attention (Shayan, Abrahamson, Bakker, Duijzer, & Van der 
Schaaf, 2015). Often these visualizations or “projections” involve 
coordinating the perceived external scene with the visualized values (Kirsh, 
2010; Szafir et al., 2016). Although novices could in principle construct these 
complex visual cues, part of expertise involves knowing which complexes 
will be helpful, and which features are likely to distract; experts quickly and 
reliably select highly relevant features, processes, and ensembles, facilitating 
later processing (Hegarty et al., 2010). In some cases, such visualizations can 
be quite complex, as when expert abacus users perform apparently functional 
gestures on imagined abaci (Hatano, Miyake, & Binks, 1977). 

Experts also create situations that are well suited to their own perceptual 
processes; part of expertise in many domains involves learning what one will 
see, and what kinds of representations might construct that seeing. Cognitive- 
anthropological approaches have long emphasized the construction of visual 


scenes and ways that cultures — both disciplinary and ethnic — may teach 


modes of seeing (e.g. Nisbett & Miyamoto, 2005). Goodwin (1994), for 


instance, notices several ways that professionals acquire and deploy 
discipline-specific ways of altering visual perception, such as circling 
important elements, trimming video clips, and directing attention to particular 
areas of potential relevance through gesture and verbal highlighting. Other 
work has emphasized the role of structured alteration to physical 
environments using psychological perception to perform distinct cognitive 


acts, such as storing or retrieving memory (Hutchins, 1995). Architects 


sketch in part to construct physically visible markers of conflict or elements 
of uncertainty in their designs. These conflicts may not be immediately 
obvious, but experts learn to construct representations that interact 
felicitously with the visual system and the typical problems presented in 
architectural scenarios (Suwa & Tversky, 1997). Chemists, likewise, must 
become skilled with a variety of representations which convey identical 
information, but each of which facilitates the perceptual processing of certain 
sorts of information (Stieff, Hegarty, & Deslongchamps, 2011). Skilled 
algebraic problem-solvers not only find it easier to read and organize 
problems when the perceptual groups are aligned with the formal rules — for 
instance finding it easier to see equivalence in the expression a*b + c*d = c*d 
+ a*b, thaninw *x+y*z=y*z+t+w * x (see Figure 10.1; Landy & 


Goldstone, 2007a), they also write expressions that embody this perceptual 


organization, facilitating the processing of these expressions (Landy & 
Goldstone, 20076). Learning to perform conceptual tasks with a high degree 
of skill often involves learning the relevant boundary conditions on the 


perceptual apparatus brought to bear on the situation. 


Manipulation Example A Acc 


PhysicalSpacing f+z * t+b = z+f * bt+t 38% 

Common Region v+th* 7 +a = h+v*a+7 23% 
; ee 0 ee e~é 

Connectedness n+b*m+#t = n+em* bet 12% 
ee 0 ee e~é 

Alphabetic Spacing at b * xs y b +a * y xX. 4% 

Structure (c¥c) + (£*£) * (9+0) + (8+k) = (f*f) + (c*c) * (8+k) + (9+0) 4% 


Figure 10.1 Example manipulations used in Landy and Goldstone (2007a), 


demonstrating the impact of visual structure on algebraic manipulation. 


Perceptual expertise is often useful or strategic, but it is rarely strictly 
necessary to complete skilled conceptual tasks. Alternative strategies are 
generally available, so that the same task can be performed using either 
efficient perceptual or perceptual-motor strategies, visuospatial reasoning, or 


largely non-visual reasoning which uses perception and action as transducers 


of internally executed computations (Clark & Chalmers, 1998; Kirsh & 
Maglio, 1994; Stieff & Raje, 2010). In such cases, different strategies will 


parcel out different tasks to perception systems, action systems, and internal 
computations, though of course any skilled behavior will involve both action 
and perception. One open question is whether perceptual-motor processes can 
trade off with aspects of expertise other than procedures (e.g. other than 
categorization or transformation between representations). The most radical 
position is that in some cases expertise allows perceptual or perceptual-motor 
processes to supplant central executive functions or even conceptual content 
(Goldstone, Landy, & Son, 2010; Kellman, Massey, & Son, 2010; Landy et 
al., 2014; Zhang & Norman, 1994). On this account, building these 


perceptual-motor routines simplifies working memory load, which is a 


typical bottleneck in high-level performance (Beilock & Carr, 2005). For 
instance, object-based attention can replace the functionality of a declarative 
recollection of one’s step in a compound process. Patsenko and Altmann 
(2010) argued that visual attention was used in this way for high-performing 
solvers of Towers of Hanoi problems. Towers of Hanoi requires for its 
operation the execution of an often quite long sequence of operations. 
Somehow, a reasoner must know where they are in the operation. One 
solution is to use internal memory resources to track the current step; 
Patsenko and Altmann, however, found that many reasoners relied on the fact 
that executing particular operations reliably left their visual attention focused 
on a particular block, so that the state of visual attention itself could replace 
an internal marking of the algorithm. 

Such Rigged Up Perception-Action Systems, or RUPAS (Goldstone et 
al., 2010; Goldstone, Weitnauer, Ottmar, Marghetis, & Landy, 2016) are 
involved in many routine operations in formal reasoning in algebra (Landy & 
Goldstone, 2007a) and programming (Hansen, Goldstone, & Lumsdaine, 
2013). Mature deployment of order of operations, for instance, relies on 
selective attention to high-precedence signs (Landy & Goldstone, 2010; 
Schneider, Maruyama, Dehaene, & Sigman, 2012) and to perceptual 
grouping (Kirshner, 1989; Kirshner & Awtry, 2004; Landy & Goldstone, 
2007a; Marghetis, Landy, & Goldstone, 2016). In particular, fMRI results 


have indicated that perceptual regions associated with grouping are highly 


activated when mature reasoners solve syntactically complex arithmetic and 
algebra problems (Maruyama, Pallier, Jobert, Sigman, & Dehaene, 2012); 
eye-tracking results likewise indicate early and automatic shifts in attention 
toward high-precedence operations (Landy, Jones & Goldstone, 2008). 


Despite early suggestions that visual strategies might be automatic (Kirshner 


& Awtry, 2004), recent evidence suggests that when students first encounter 
syntactically complex expressions such as 4+3x9, they rely on deliberate, 
strategic shifts of attention to problem subcomponents, and that reliance on 
grouping and automatic attention comes only with extended practice 
(Braithwaite et al., 2016). 

According to RUPAS, expertise involves not simply becoming more 
efficient or strategic in perceiving, but actually using perceptual processes for 
novel purposes, essentially co-opting through notations and inscriptions the 
typical role of the perceptual process. To some degree this graphical co- 
option of the conceptual is obvious: plotting our designs as line-plots allows 
the detection of inclined lines to proxy for the detection of main effects and 
interactions in psychological experiments. The specific claim of the RUPAS- 
style account is that co-option of resources applies at the level of cognitive 
process: attention replaces short-term procedural memory and visual binding 
replaces the application of complex relational rules (Maruyama et al., 2012). 
At the same time, this co-opting may impact visual perception itself, at least 
in targeted situations. For instance, in a classical Eriksen flanker task, 
Goldstone, Landy, and Son (2010) found that simply detecting addition signs 
was interfered with by the presence of surrounding multiplications — 
suggesting that attention was drawn to the high-precedence operation, and 
more so than the reverse. Marghetis et al. (2016) report that visual grouping 
is itself impacted by formal syntax. In this task, a compound expression is 
presented to subjects, who have then to detect a property over two elements 
of the expression. For instance, in the expression “a + b*c + d,” the b and c 
might change colors. The participants’ job is to detect the color change, and 
respond to whether the new colors match. Crucially, participants are better at 


doing this when the changing objects lie across high-precedence than low- 


precedence operations, suggesting that their visual representations may have 
bound these terms into objects (cf. Zemel, Behrmann, Mozer, & Bavelier, 
2002). 

In short, learned perceptual and perceptual-motor routines play a strong 
role in the acquisition of expertise. These routines often involve shifting 
attention, mental rotation, visualization, and the extraction of particular 
features. Perceptual-motor routines must be aligned with existing 
representations, and learning skills involves’ learning appropriate 
representations and their relationship to existing or acquirable perceptual 
processes. These routines may even replace quintessentially non-perceptual 
content; furthermore, the routines acquired may generalize beyond the 
specialized representations and situations that mandate their creations, to 
affect perception in a variety of situations. In the next section, we turn to our 
second question, the development of specialized or distinctive neural systems 


instantiating perceptual categories. 


Development of Specialized Functional 
Networks 


Decades of vigorous debate have resolved themselves into a consensus view 
— that the occipital lobe undergoes substantial neural reorganization in 
response to long-term exposure to situations involving complex visual 
categories, and that experience drives at least some of this organization. The 
so-called fusiform face area provides an instructive example case. Originally 
identified as a region especially involved in processing facial geometry 


(Kanwisher, McDermott, & Chun, 1997), more recent work has fairly 


persuasively identified a region of the fusiform gyrus as broadly supporting 


expert processing of visual category learning beyond faces (Bukach, 


Gauthier, & Tarr, 2006), as part of a collection of distributed networks that 


subserve object perception (McGugin, Gatenby, Gore, & Gauthier, 2012). 
Activation of the fusiform face area during perception of a category exemplar 
increases with experience and skill with the category, for a wide range of 
visual categories, including cars, birds, artificial “greebles,” and perhaps 


musical notation (Gauthier et al., 2000; Gauthier, Behrmann, & Tarr, 1999; 


Wong & Gauthier, 2010). Whether different kinds of expertise recruit the 


Same precise neurons, or whether each kind of expertise recruits a slightly 
differentiated sub-area within the FFA is still under dispute, as is whether and 
how faces remain especially well suited to the FFA (Grill-Spector, Sayres, & 
Ress, 2006; McGugin et al., 2012). One leading hypothesis compatible with 


both possibilities is that neural regions are recruited into functional networks 


based on the processes they perform, rather than the content domain they 


cover (Anderson, 2014). However, in some cases increased activation of the 
FFA has not been found (Brants, Wagemans, & Op de Beeck, 2011; Grill- 
Spector, Knouf, & Kanwisher, 2004; Op de Beeck, Baker, Dicarlo, and 
Kanwisher, 2006), raising concerns over the generality and robustness of the 
connection between the FFA and expert behavior. 

This is perhaps most strikingly demonstrated in the case of the so-called 
visual word form area, a functionally identified region in the fusiform gyrus 
(Cohen et al., 2000; McCandliss, Cohen, & Dehaene, 2003). Because writing 


is a very recent culturally specific invention, it is unlikely that we have novel 


dedicated processing regions for word forms (unlike faces). Nevertheless, the 
network of activation in literate adults reliably involves a brain region 
adjacent and relatively anterior to the fusiform gyrus — the visual word form 
area. Although this region is involved in many different tasks (as is nearly 
every part of the brain; Anderson, 2007; Poldrack, 2011), the region is 
selectively active in response to symbols and words, and belongs to a 
network selectively responsive to words (Reinke, Fernandes, Schwindt, 
O’Craven, & Grady, 2008). Though the exact role of the visual word form 
area remains unclear, it seems that becoming expert readers involves the 
selective recruitment of a particular and content-appropriate neural substrate, 
and that that substrate bears both strong similarities to and _ striking 
differences from the network recruited for other domains of perceptual 
expertise. 

An open question is this: if written language has not evolved dedicated 
neural processing regions, why does it reliably capture a particular section of 
visual cortex? What properties does that particular region have that are so 
uniformly useful for word detection? One possibility is that the prevalence of 


boundaries in the natural visual environment supported the evolution of 


relatively specialized boundary and intersection detection systems, and that 
written languages so resemble boundary intersections that they can recruit 
these powerful evolved systems. In line with this thesis, Changizi measured 
the prevalence of vertex types in a variety of languages, and found a close 
match to the vertexes created by intersecting boundaries in natural scenes 
(Changizi, Zhang, Ye, & Shimojo, 2006). This raises in turn the further 
possibility that languages themselves adapt to better suit the learning biases 
of the neural mechanisms they co-opt. Though speculative, the possible co- 
adaptation of neural mechanisms and cultural targets supporting expertise is 
one of the most promising and forward-looking directions for research into 
perceptual expertise. 

A second (non-exclusive) possibility is that the visual word form area 
lies along an important conduit connecting perception of visual objects and 
their construction (James, 2010; Li & James, 2016). This account emphasizes 
the connection between perception and action, and the role that each plays in 
constraining and informing the other. Taken together, these results emphasize 
the redeployment and co-option of existing perceptual regions in the service 
of perceptual expertise, and the development of specialized networks, rather 
than specialized, task-specific neural regions. Over the next several years, we 
can expect more detailed examinations of the structure and development of 
particular networks that serve expert-level perception, and especially the 
potential interactions between action and _ perception (e.g. Lake, 
Salakhutdinov, & Tenenbaum, 2015). 

Regardless of the fate of the fusiform face and visual word form areas, 
per se, it is now clear that visual cortex broadly reorganizes itself in 
processing highly trained perceptual categories. Many different regions show 


expertise-related changes, including the parahippocampal place area and the 


precuneus (McGugin et al., 2012; see Harel, Kravitz, & Baker, 2013 for a 
review). Furthermore, task goals impact visual cortex representations of 


objects (Harel, Kravitz, & Baker, 2014), suggesting an interaction between 


attention, intention, and visual representation. 


(How) Does Expertise Change Low-Level 
Perceptual Experience? 


One of the most fascinating and obvious questions in cognitive psychology is 
“Does what we think change what we see?” Despite over 100 years of 
scientific investigation, no scientific consensus exists on an answer to this 
question. In 1910, Perky reported that imagined objects interfered with 
perception, and vice versa, so that an imagined banana interfered with the 
detection of real patches of yellow illumination, while those color patches in 


turn might impact imagined bananas (Perky, 1910; see also Segal & Nathan, 


1964). Since that time, people have argued that experience obviously could 
not affect low-level perception (Fodor, 1984), that it demonstrably does 
(McClelland, Mirman, Bolger, & Khaitan, 2014; Proffitt & Linkenauger, 


2013), and that evidence for any such effect is unclear at best (for recent 


reviews of the controversy, see Firestone & Scholl, 2015; Lupyan 2015). We 


will resolve none of this here. Instead, we will consider the simpler question 
of when and how expertise seems to impact perceptual performance; that is, 
does expertise with a category or topic shift how people can behave (leaving 
aside the question of what they actually “see”)? Even within this more 
reduced question, though, the impact of perceptual experience on 
performance is hardly uncontroversial. 

Behaviorally, several effects of expertise in a visual category have been 
extensively studied, comprising inversion costs, holistic processing, and 
advantage (or equality) of the individual over the basic level. By no means is 


this list exhaustive; indeed, several of these advantages were largely 


documented first in face processing literatures, and later documented for 
other real-world areas of expertise. 

Inversion costs refer to diminished discriminability of inverted as 
opposed to upright objects within a well-known category (see Figure 10.2). 
First demonstrated in faces (Yin, 1969), this phenomenon as a property of 
expertise was originally proposed by Diamond and Carey (1986) in dog 
experts, who were better able to remember and recognize images of dogs 
within their breed of expertise when the images were presented upright, as 
opposed to inverted. The same is not true of novices, or generally even of 
experts outside their specific category of expertise. Since then, the inversion 
effect has been documented with experts in a variety of specific visual 


categories, including birds, cars, and artificial “greebles” (Gauthier et al., 


2000), and is stronger in own-race faces than other-race faces (Rhodes, 
Brake, Taylor, & Tan, 1989). 
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Figure 10.2 Upright and inverted faces and cars. Faces are more easily 
discriminated by typical populations when presented upright. This effect is 
diminished for other familiar categories, but occurs for category-specific 
experts (e.g. dog breeders, car experts). 


A second major phenomenon associated with perceptual expertise is 
holistic processing. Behaviorally, heuristic processing refers to a failure of 
selective attention — the ability to isolate and respond to particular features of 
a stimulus. In the familiar case of faces, one holistic marker is that people are 
impaired at making same—different judgments on single features, such as 
eyes, when those eyes are embedded in a face (Hole, 1994; Richler, Cheung, 
& Gauthier, 2011; Rossion, 2013; Young, Hellawell, and Hay, 1987). More 


typically, a face is separated into top and bottom halves, which may 


independently change or stay the same on any trial. Although the effect is 
quite powerful, and even interferes with the identification of familiar faces 
(Young et al., 1987), it is somewhat sensitive to methodology (Konar, 


Bennett, & Sekuler, 2010; Richler et al., 2011). More theoretically important, 


the effect is almost entirely disrupted by small physical misalignment of the 
top and bottom face halves, suggesting that holistic processing is driven by a 
highly template-like knowledge of facial geometry (Figure 10.3). As with 
inversion, this paradigm has also been applied to experts in a range of 
categories, with the general finding that experts process visual images 
holistically, at least to some degree (Bukach, Phillips, & Gauthier, 2010; 
Tanaka, Kiefer, & Bukach, 2004; Wong, Palmeri, & Gauthier, 2009; but see 


McKone, Kanwisher, & Duchaine, 2007). As with inversion, holistic 


processing seems to be less powerful with other races than with one’s own 
(Tanaka et al., 2004). 


Aligned Misaligned 


eS. 


Figure 10.3 Holistic processing of faces is disrupted with just a small 
misalignment of upper and lower face halves. This pattern has also been 


replicated in other types of perceptual expertise. 


Finally, for a very large variety of categories, a so-called “basic level” 
can be identified (Rosch, 1978). One classic marker of this basic level is that 
identification at that basic level is fast and accurate — more so than 
subordinate level classification, in particular. For experts in a category, 
however, identifying individuals and sub-types is often privileged instead 
(Johnson & Mervis, 1997; Tanaka, 2001). Furthermore, pairs of items that are 
conceptually connected may be visually processed differently than equally 
similar item pairs that are conceptually disconnected. Lupyan (2008) reports 
that visual search was facilitated when distractors were upper- and lower-case 
versions of the same letter (e.g. B vs. b) than when they were different, 
visually similar letters (e.g. B vs. p), suggesting that experience with category 
labels affects distractor processing. 

There are, of course, many low-level perceptual processes that may be 


touched by expertise; this review covers only a small sample of those that 


have been extensively studied by cognitive neuroscientists. One fascinating 
question is how disciplines that include accurate perception and 
reconstruction of visual or physical scenes as part of the disciplinary craft 
may alter perceptual practices — for instance, visual artists often rely on an 
ability to accurately depict (and of course, to sometimes deliberately 
inaccurately depict) real or imagined visual scenes. Artists are indeed 
demonstrably better at a number of visual tasks, such as selecting key lines 
and curves of a scene that maximally facilitate identification, selecting 
patterns that indicate or suggest textures, and guiding visual attention. 
However, evidence regarding low-level or sweeping changes to visual 
perception in visual artists has been equivocal, at best, and perhaps 
incompatible with a strong hypothesis of radical visual training in the service 
of expertise. For a review of these and other topics related to artistic 


expertise, see Kozbelt and Ostrofsky (Chapter 30, this volume). 


Conclusions 


Interactions between perception and expertise take many different forms. 
Visual perception organizes to facilitate the identification of important 
features, sub-types, and structural geometries of expertise-relevant categories. 
Over long-term training, specialized networks adapt to handle such visual 
categories. Additionally, however, research in cognitive science is just 
beginning to uncover myriad ways that external representations, generic 
perceptual processes, and learned complex perceptual routines combine to 
support general expertise. In the limit, complexes of temporally extended 
perceptual routines over well-designed diagrams and notations may 
supplement or even replace what might have been more abstract and 
conceptual reasoning processes — a process that seems to unfold over the 
course of acquiring significant skill in a particular domain or task. 

Over the next decade, we expect two open questions to be the focus of 
especially active research. First, we predict that specific neural complexes — 
processing networks that span the whole brain — will be mapped for specific 
frequent simple object categories that people commonly attain expertise with, 
especially letters and faces. Second, we predict an increasing emphasis on 
building rich behavioral understandings of real-world perceptual and spatial 
processes, and the role they play in highly skilled behaviors, especially across 
the STEM disciplines (science, technology, engineering, and mathematics). 
Detailed cognitive traces of extended behavioral perceptual-motor routines 
involved in complex actions such as transforming molecules between 


representations to answer conceptual questions may, we hope, guide the 


exploration of the neural and environmental structures implementing these 


complex but important roles played by perception in real-world expertise. 
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Introduction 


Knowledge is generally regarded as that body of facts, information, 
descriptions, understandings, and skills possessed by an individual, a team, 
an organization, or a social group. Knowledge can be about something 
(factual) or about how to do something (skillful) and may be explicit 
(accessible to conscious awareness) or implicit (not readily accessible to 
conscious awareness). In this chapter, we discuss knowledge elicitation, 
knowledge representation, and cognitive analysis as employed in the field of 
Cognitive Systems Engineering. 

Cognitive Systems Engineering can be thought of as having emerged 
from traditional human factors engineering. It is a professional discipline that 
uses methods of cognitive analysis to ensure that cognitive work in socio- 
technical work systems is both efficient and robust. The aim is to amplify and 
extend the human capability to know, to perceive, to decide, to plan, to act, 
and to collaborate by integrating technical functions with the cognitive 
processes they need to support. Design targets include software and 
hardware, training systems, organizations, and workspaces. 

Knowledge elicitation and knowledge representation are central to the 
Cognitive Systems Engineering enterprise. One goal for this chapter is to 
outline how methods of knowledge elicitation and knowledge representation 
have advanced, illustrate recent and salient findings, and suggest emergent 
directions for research and theory. Indeed, there have been significant 
advances since the first edition of this handbook (see Hoffman & Lintern, 


2006). First, we briefly recount the historical background. 


Cognitive Systems Engineering 


Cognitive Systems Engineering evolved from research that seemed at odds 
with the standard cognitive paradigm as inspired by information processing 
theory and the computer metaphor that was foundational in the emergence of 
cognitive psychology in the 1970s—80s. That standard cognitive paradigm 
asserts that human cognitive activity, to be effective, should follow a formal 
process of reasoning (deduction, inference, or interpretation from premises, 
facts, or principles) (e.g. Kahneman, 2011). Cognitive Systems Engineering 
takes a naturalistic perspective, making no such presupposition about the 
nature of human cognition, preferring instead to engage in research in natural 
(workplace or field) settings to reveal the nature of cognition. 

Naturalistic inquiry within dynamic, high-intensity work settings has 
revealed that workers rely on a mix of formal, informal, and affective 
cognitive processes (Klein, 1998) and further, that the imposition of formal 
decision strategies can actually interfere with expert cognitive performance 
(Beach & Lipshitz, 1993; Zakay & Wooler, 1984). In addition, experts are 


less likely to rely on formal cognitive processes than are novices (Klein, 


1989), suggesting a progression during the development of expertise from 
formal to informal cognitive processing rather than the inverse as would seem 
to be implied by the standard cognitive paradigm. 

Problematically for design of cognitive support systems, detailed and 
systematic analyses of natural work settings typically show that a focus on 
explicit and structured aspects of cognition can overlook important work 


processes (e.g. Cook, Woods, Walters, & Christoffersen, 1996). Experienced 


workers deploy subtle strategies that are sometimes surprising, even 
counterintuitive, defying the logic of formally structured analysis (Klein, 
1998). Often, cognitive tools developed from a formal analysis not only fail 
to support those subtle strategies, but worse, they contain features that are 
difficult or even impossible for workers to deploy (e.g. Cook et al., 1996). 
The resulting work practice may appear to be more efficient but will often be 
less effective and less robust, particularly in unusual or challenging 
situations. 

Cognitive Systems Engineering subscribes to the view that if we are to 
design technologies that effectively support cognitive work, we need to 
understand the goals and cognitive challenges of that work. Anthropological 
studies of human work (Hutchins, 1995; Jordan, 1989; Lave, 1988; Lave & 
Wenger, 1991) as well as naturalistic studies of decision making (Crandall & 
Getchell-Reiter, 1993; Klein & Calderwood, 1991; Pliske, Crandall, & Klein, 


2004) have shown that effective workers resolve challenges in novel and 


creative ways. 

Cognitive Systems Engineering draws on several theoretical or 
conceptual paradigms (Militello, Dominguez, Lintern, & Klein 2010), where 
the notion of paradigm should be understood as it is in philosophy of science: 
an organized set of interrelated theories (laws, concepts) and research 
methods. A paradigm for Cognitive Systems Engineering serves to guide 
knowledge elicitation and knowledge representation by identifying the 
cognitive-relevant structures and processes to be targeted and indicating how 
those structures and processes are related to each other. The selected 
paradigm will suggest what information is to be sought during knowledge 
elicitation and how that information might be summarized, analyzed, and 


depicted within a knowledge representation. 


Despite noteworthy differences between the dominant Cognitive 
Systems Engineering paradigms, their common features are more relevant to 
our discussion in this chapter. All rely for knowledge elicitation on some 
form of cognitive interview, document analysis, observation of work 
performance, or stakeholder workshop. Individuals having backgrounds in 
other professions (e.g. commercial aviation, nursing, anesthesiology) have 
been able to reflect on their own work experience as a substitute for 
knowledge elicitation. Additionally, all Cognitive Systems Engineering 
paradigms rely on one or more strategies of knowledge representation: tables, 
flow diagrams, node-and-link diagrams, functional hierarchy maps, or text 
summaries. 

While the conceptual models associated with each of the Cognitive 
Systems Engineering paradigms are different, all are largely 
phenomenological in the sense that they are motivated by detailed 
descriptions of mental activity as provided by individuals who engage 
actively in that work. For example, knowledge elicitation for Decision- 
Centered Design can result in a narrative that can be mapped onto the 
Recognition-Primed Decision Model of Klein (1998). Similarly, knowledge 
elicitation for Cognitive Work Analysis may result in a narrative that can be 


mapped onto a Decision Ladder (Lintern, 2010). These models offer succinct 


and evocative depictions of the ideas that establish the associated strategy as 
a paradigm and point to the characteristics of the knowledge sought in the 
knowledge elicitation phase. 

Cognitive Systems Engineering models should not be taken as 
theoretical statements that imply hypothetical constructs with a physical or 
objective reality having properties that go beyond those identified in the 


phenomenological narratives, as does, for example, the ubiquitous 


information processing model. Rather, they offer modal descriptions of 
cognitive workflows and supporting work structures as described by highly 
proficient, highly experienced, or expert workers. These models are, 
however, based on the theoretical assumption that cognitive work involves 
perceptions, states of awareness, and a variety of cognitive processes. For 
example, the Recognition-Primed Decision Model appeals to the cognitive 
process of mental simulation as a means for evaluating a proposed course of 
action. Similarly, the Decision Ladder identifies the cognitive states and 
cognitive processes assumed to be involved in cognitive work and further 
implying that these states and processes constitute the complete set of states 


and processes that can be involved in cognitive work. 


Expertise 


Of particular interest for Cognitive Systems Engineering is cognitive work 
that is performed with high levels of proficiency. Thus, one of the goals of 
Cognitive Systems Engineering is to understand the nature of expertise as it 
reveals itself in work practice. However, it remains something of a challenge 
to identify those workers who can be characterized as experts, at least in 
advance, because cognitive engineers do not often know what constitutes 
expertise in any particular domain until they have elicited knowledge from 
that domain. Furthermore, in many cases, there are few readily available 
objective performance criteria that can be measured in work settings. One 
obvious criterion, years of experience, is useful and informative, but does not 
conclusively predict expertise because length of domain experience does not 
necessarily translate into proficiency (Ericsson, Whyte, & Ward, 2007; 
Hambrick, Ferreira, & Henderson, 2014). 

It is nevertheless possible to define expertise on the basis of abilities, 
capacities, or skills that set some workers apart from others: key indicators 
that are known to be associated with high standards of performance 
(Hoffman, Ziebell, Feltovich, Moon, & Fiore, 201 1c). In general, as research 
suggests, experts see things that non-expert workers do not. Experts are also 
better at taking the perspective of others, at anticipating the consequences of 
actions, and at improvising and adapting when facing unusual situations. In 
comparison to other workers, experts form richer understandings of 


situations, spot more safety concerns, and form better plans. These are the 


types of abilities that can be used to distinguish experts from lesser skilled 


workers such as apprentices, novices, and journeymen. 


Knowledge Elicitation 


Within Cognitive Systems Engineering, knowledge elicitation is used to 
identify the knowledge employed by practitioners to perform useful cognitive 
work. The method of knowledge elicitation to be used must be suited to the 
particular type of knowledge to be elicited; whether it be factual of skillful, 
explicit or implicit, or held by individuals, teams, organizations, or social 
groups. Knowledge elicitation methods include observation, interview, and 
document analysis. 

While the elicitation strategies are essentially the same for individuals, 
teams, organizations, or social groups, the knowledge elicitation protocol will 
focus on different knowledge elements for each of these entities and as 
consistent with the particular cognitive model held by the project staff. For 
example, Figure 11.1 shows models for individual cognition (top panel) and 
team cognition (bottom panel). The team model in Figure 11.1 was adapted 
from Klein (1998) and the individual model was developed for this chapter 
with elements drawn from that team cognitive model and the Recognition- 
Primed Decision Model also shown in Klein (1998). As is evident from 
inspection of these two models, the general character of the knowledge to be 
elicited in analysis of individuals versus teams is similar but differs in many 


of its specific properties. 
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Figure 11.1 Cognitive models for individuals (top panel) and teams 


(bottom panel). 


Knowledge elicitation can be challenging when the focus of 
investigation is on a team or organizational unit. It is more difficult to 
interview a group than an individual, and researchers will often select and 
schedule individuals from a team for separate interviews. Nevertheless, 
whole team interviews offer opportunities for team members to clarify issues 


on the spot and they occasionally prompt team members to come to an 


explicit consensus on concepts or issues that had previously been left implicit 
(e.g. Hoffman, Schraaagen, van de Ven, & Moon, 20115). 

Workplace observation raises similar challenges. Because observation of 
a team from only one point of view does not reveal how the team makes 
decisions as a whole, teams should be observed from different, strategically 
selected points of view. Especially if observations of a whole team from 
multiple points of view can be integrated into a coherent representation, this 
offers the possibility of unique insights. For co-located team members, a 
single observer can monitor the activities of several team members by 
alternating their attention across important members of the team or by 
following the flow of information or responsibility. A workable strategy for a 
large team is to have several observers with assigned responsibilities. For 
team members working in separate locations, several observers are essential, 
with each being assigned to selected high-value locations. 

Knowledge elicitation can reveal implicit knowledge by scaffolding 
workers to help them articulate cognitive processes and skills they employ, 
perhaps even without conscious awareness. Special knowledge elicitation 
strategies have been developed for that purpose, most notably the Critical 
Decision Method (Hoffman, Crandall, & Shadbolt, 1998; Klein, Calderwood, 
& MacGregor, 1989) and Concept Mapping (Crandall, Klein, & Hoffman, 


2006). Beyond that, there is a need for systematic elicitation methods even 


for explicit knowledge; an unsystematic effort (such as an unstructured 
interview) will likely miss important elements. Accordingly, in addition to 
uncovering implicit knowledge, knowledge elicitation has a rather prosaic 
bookkeeping function. 

It is always a challenge to elicit the right knowledge: those knowledge 


elements that advance a project toward its goals. No particular method will 


uncover all critical implicit knowledge or document all critical explicit 
knowledge. In any project, cognitive engineers should remain mindful that 
there can be no proof of completeness, in part because the corpus of 
knowledge in any work setting does not have defined or even fixed 
boundaries. The stopping point for any knowledge elicitation effort must rely 
on the cognitive engineer’s professional judgment. Possibly the best 
assessment of sufficiency will be based on the effectiveness of any resulting 
product, the design of which has been guided by the elicited knowledge. 

In proceeding with knowledge elicitation, cognitive engineers need to be 
sensitive to the conceptual frame of the workers whose knowledge is to be 
elicited. While project documents and design recommendations might be 
framed in cognitive-theoretic language, cognitive engineers should avoid 
such language in their interactions with workers; ideally, those interactions 
should use the natural expressions and concepts employed by workers (e.g. 
Regan, Lintern, Hutchinson, & Turetschek, 2014). Further, cognitive 
engineers need to avoid the temptation to confirm their own beliefs as they 
frame questions and interpret responses. They also need to recognize and 
make sense of the diverse attitudes and beliefs to be found in any work 
community or, in other words, they need to take account of differing 
stakeholder perspectives. While many projects will require elicitation of 
knowledge from a range of stakeholders (e.g. in healthcare, possibly general 
practitioners, medical specialists, nurses, hospital support staff, managers, 
and information technologists), those diverse attitudes and beliefs will be 


evident not only between stakeholder groups but also within them. 


Knowledge Representation 


The immediate products of knowledge elicitation include notes, diagrams, 
and recordings. Notes may capture a narrative, responses to specific 
questions, and the cognitive engineer’s thoughts. Diagrams may depict 
timelines, explanations, and maps. 

The goal for knowledge representation is to organize the elicited 
knowledge in a manner that will at least facilitate understanding and, more 
commonly, also facilitate design or redesign, with the ultimate aim of 
supporting work practices that are both effective and robust. The essential 
knowledge elements, those most pertinent to the goals of the project, may be 
abstracted within a text summary, a table, or a node-and-link diagram that 
represents one or more of activity flow, cognitive states, cognitive processes, 
concepts, relationships, or structure. For example, the individual and team 
models of Figure 11.1 depict concepts that underlie cognition. The linking 
arrows imply interactions between concept categories but the nature of those 
interactions remains unspecified in these two models. 

Some methods of cognitive analysis blend knowledge elicitation with 
representation. Concept Mapping, for example, includes protocols for 
elicitation that enable a cognitive engineer to co-create the representation in 


collaboration with workers (Moon, Hoffman, Novak, & Cafias, 201 1a). 


Design 


Within Cognitive Systems Engineering, where the focus is on cognitive 
issues, strategies for design have not always received the attention they 
deserve. Unfortunately, interface design and human systems integration 
largely continue to remain the province of information technology specialists 
and engineers who typically have limited sensitivity to cognitive issues. 
Nevertheless, there is an emerging trend within Cognitive Systems 
Engineering to be more explicit about design implications. A Decision 
Requirements or Cognition Requirements Table developed within a 
Decision-Centered Design project (e.g. Table 11.1) may include a column for 
design ideas (Crandall et al., 2006; Klein, Militello, Dominguez, & Lintern, 
in press). Similarly, the elicitation and representation effort within a 
Cognitive Work Analysis project may use the structured processes of 
Functional or Ecological Interface Design (Bennett & Flach, 2011; Lintern, 
2013) or a design toolkit (Read, Salmon, Lenné, & Jenkins, 2015) for 


translating the outputs of cognitive analysis into concrete designs. 


Table 11.1 Cognition Requirements 
Anesthetist (Patient struggling to breathe) 


Case 


Difficulties 


Critical Features 


Critical Expertise 


Design Ideas 


An anesthetist, when called to the 
Emergency Department to see a 
patient with breathing difficulties, 
observed the patient's low oxygen 
saturations. The anesthetist attempted 
ventilation with a bag-mask. The 
patient’s oxygen levels did not 
improve. The anesthetist then 
attempted to insert a breathing tube, 


A swollen airway 
concealed the 
anatomical 
landmarks 
normally used to 
guide tube 
insertion. 


It was not initially 
evident that the 


Swollen airway 


Normal 
anatomical 
landmarks not 
visible 


Know alternate strategies 
& when to use them 


Know when to abandon a 
failing strategy; novices 
might persist, putting the 
patient at additional risk 


Know how quickly oxygen 
levels change if bag-mask 


A greater focus 
on training 
difficult cases 


Improve the 
laryngoscope or 
design a system 
to guide 
breathing tube 
insertion into a 


using a laryngoscope to guide the 
tube past the patient's pharynx. This 
also failed. The anesthetist declared 
‘failed intubation and failed ventilation’ 
and decided to open the airway 
surgically with a needle. 


patient's airway ventilation is working 


patient with a 
was swollen. 


challenging 
anatomy? 


Know success trajectory 
of breathing tube 
insertion; recognize when 
to discontinue attempt 


The added value of any particular analysis for design can always be 
questioned. Does the analysis help us understand work practices and 
challenges in a manner that leads to better design ideas, or would we have 
been just as well off by relying on our own informal appraisal of what is 
needed? We cannot expect any particular design strategy to specify the 
design solution, but it should at least guide the transition from analysis to 
design in a productive direction by stimulating promising ideas while helping 
to eliminate bad ones (Bennett & Hoffman, 2015). The new design should be 
demonstrably better as a result of the analysis in relation to what we might 
have accomplished just by allowing our imagination free rein following 


unsystematic appraisal of the work. 


The Challenge 


Cognitive Systems Engineering emerged in part as a response to the neglect 
shown by the technology-centric design disciplines and also by mainstream 
cognitive science to the cognitive constructs that are implicated in human 
work. In particular, technology-centric design disciplines ignore human 
cognition while mainstream cognitive science maintains a commitment to, 
indeed, an exclusive focus on, preconceived notions of mental constructs 
drawn from the information processing hypothesis of the mind. The techno- 
centric mindset that justifies each of these approaches is responsible for many 


of the problems that afflict modern cognitive support systems. 


Illustration: The UK National Health Service 


Challenger, Clegg, and Shepherd (2013) offer insight into the problems that 
result from designing cognitive support systems based on a techno-centric 
view of work practice. They describe the experience of the UK National 
Health Service with an electronic healthcare record known as the National 
Health Service Care Records Service. This system was developed to manage 
medical records for all patients in the UK National Health Service. The goal, 
laudable in itself, was to ensure that every patient’s healthcare information 
would be integrated into a single record, which could then be accessed at any 
location within the UK National Health Service. 

Despite the many obvious benefits of computerization, there was 
widespread evidence of diverse problems: a lack of compatibility with 
clinical practice, incomplete and inaccurate information, a restrictive data 
entry strategy, and an electronic-notes function that increased the cognitive 
work associated with taking a patient history, to name just a few. There was 
little evidence of the anticipated benefits. 

From a Cognitive Systems Engineering perspective, the sorts of 
problems experienced with the National Health Service Care Records Service 
are unsurprising. Indeed, such findings have been routinely demonstrated 
since the introduction of electronic health records (Miller, Moon, Anders, 
Walden, & Montella, 2015). The design strategy was essentially a 
technology-push driven by a political agenda. The primary goal was to 
computerize work practices. 

There was no evidence to suggest that the technological development 


was driven by a need to better support the work practices or the work goals of 


the diverse stakeholders within the system. Nor was there any evidence of 
developers considering that there are many ways to deploy computerization 
within a work environment and that some care needs to be taken to ensure 
that the chosen strategy is one of the more effective rather than one of the less 


effective. 


Illustration: Intelligence Analysis 


Moon and Hoffman (2005) found a similar response to calls for broad and 
deep change within the US intelligence community following the 9/11 
tragedy. The demand was political and the development process was largely 
techno-centric. Recommendations were offered in the form of commission 
findings that were ostensibly derived from an understanding of the way in 
which intelligence analysts go about their work. The recommendations 
included increased use of software to support argument formulation: 
automation for deriving meaningful links between entities, for sorting and 
comparing evidence, and for alerting intelligence analysts to important new 
data. 

While the recommendations offered some reasonable strategies for 
consideration, they rested on widely held and rather superficial assumptions 
about the nature of the cognitive work undertaken by intelligence analysts. A 
cognitive task analysis of intelligence work, as afforded through knowledge 
elicitation and knowledge representation, offered many counterpoints to those 
common assumptions. Multiple investigations (Hoffman, Henderson, Moon, 
Moore, & Litman, 2011a; Hutchins, 2007; Johnston, 2005) provided 


empirical evidence about the ways intelligence analysts actually go about 


their work. From this perspective, it was readily apparent that software 
technologies built to satisfy politically driven demands for change actually 
served as barriers to the cognitive process of sense-making (Klein, Moon, & 
Hoffman, 2006). 


Implications: The Way Forward 


From a Cognitive Systems Engineering perspective, a viable strategy for 
computerization will seek to support human work rather than replace it with 
technological functionality. To do that, we need to understand how the work 
is accomplished and how workers deal with their everyday complexities and 
challenges. Conceptually at least, the strategy of designing from a thorough 
understanding of the work of all stakeholders is straightforward. It is the 
strategy that the methods of knowledge elicitation and knowledge 
representation we discuss in this chapter were designed to support. In contrast 
to the techno-centric view of work, this strategy takes a naturalistic, 


ethnographic approach by seeking to abstract essential cognitive constructs 


from work narratives (e.g. Rasmussen, 1986; Klein & Calderwood, 1991). 


Illustration: Landmine Detection 


A landmine is an anti-personnel or anti-vehicle explosive device typically 
concealed under a thin layer of topsoil or debris, to be pressure detonated 
when stepped on or driven over. Injuries sustained by detonation of a land 
mine can be severe. Thus, the crucial demand is to detect landmines before 
they are detonated. The primary enabling technology for advance detection 
has historically been the metal detector, but working against successful 
detection is the trend toward development of landmines with minimal metal 
content. Staszewski (2004) reported a detection rate of approximately 20 
percent for low-metal anti-personnel mines (Figure 11.2, left panel, before 
training). 
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Figure 11.2 Landmine detection with and without cognitive training 
(adapted from figures 1 and 2 of Staszewski, 2004). Error bars show 


estimated 95% confidence intervals. 


In an effort to improve detection rates, Staszewski (2004) interviewed 
two military experts whose detection rates exceeded 90 percent (well above 
the 20 percent norm). He elicited from these experts a description of how 


they achieved their exceptional performance, discovering that they used 


special strategies to enhance the sensitivity of their equipment and for 
information search and decision making. In particular, he discovered that 
these two experts synthesized spatial patterns by attending to the onset and 
offset of auditory signals that accompanied sweeping motions of the detector 
head. They then mentally compared these synthesized patterns to patterns 
they had learned were produced by mines. Staszewski developed a 20-hour 
program to teach these strategies to other soldiers, who were then able to 
detect low-metal anti-personnel mines at the expert rate of over 90 percent 
(Figure 11.2, left panel, after training). 

While this is a powerful demonstration of the potency of this sort of 
cognitive training, the work was seemingly made irrelevant by an 
independent engineering development of a new type of metal sensing device. 
The new device used two sensors in conjunction with improved algorithms. 
However, although it was developed to fix those abysmal detection rates for 
low-metal mines, detection rates with the new device were actually worse 
than with the old device (Figure 11.2, right panel, before training). To resolve 
this problem, Staszewski adapted the new training program for the old device 
to be suitable for the new device. Following training on this adapted program, 


detection rates again improved to over 90 percent. 


Illustration: Emergency Response Teams 


Klinger and Klein (1999) reported a cognitive design effort focused on 
improving the performance of an emergency response team. The project was 
initiated by an emergency planning director within a nuclear power facility 
who was concerned that his response team was not performing well during 
certification exercises. In particular, it seemed that everyone in the response 
team was overloaded during an exercise. The planning director expressed the 
view that the overload would need to be resolved with labor-saving 
technology because the emergency control room was already crowded, 
thereby precluding the possibility of adding staff. 

Several cognitive engineers observed an exercise in progress and, 
following the exercise, interviewed a number of the staff. They identified a 
high number of delays due to hand-offs and a number of information 
bottlenecks. They observed that the flow of information to key decision- 
makers was inefficient and that a number of the staff had a poor appreciation 
of their own roles, of who made key decisions, and how information was 
supposed to flow through the team. 

The cognitive engineers recommended that the room layout be 
reorganized to put people who interacted with each other closer to each other. 
They further recommended that team leaders clarify team member roles and 
explicitly identify those who would make the key decisions, and that the team 
make better use of after-action reviews. Another key recommendation was to 
consolidate staff positions, which resulted in a substantial reduction in staff 
numbers from 80 to 35 (Militello et al., 2010). The only hardware 


recommendation was decidedly low-tech; white boards as shared displays to 
maintain common ground. 

All recommendations were implemented, and the next exercise ran 
much more smoothly. There was far less noise and confusion. Staff members 
asked fewer questions because their situation awareness had improved. Key 
decision-makers were able to expand their time horizons and found 
themselves thinking ahead instead of reacting to problems. Paradoxically, 
workload decreased dramatically despite the reduction in staff. The workload 
problem was resolved not by adding staff or inserting supposed labor-saving 


technologies but by reallocating the work into more efficient work packages. 


Commentary on the Techno-Centric Mindset 


Even in areas where knowledge elicitation and knowledge representation 
tools could potentially make a huge difference, their full acceptance in the 
design, engineering, and management communities remains elusive. Possibly 
the most significant challenges faced by cognitive engineers is in countering 
the techno-centric attitudes that ignore the power of human expertise and in 
gaining acceptance of the knowledge acquisition and representation methods 
within domains in which they could make a huge difference. 

Within this context, the studies by Stazewski (2004) and Klinger and 
Klein (1999) do double duty. On the one hand, they demonstrate the power of 
thoughtful application of knowledge elicitation and knowledge representation 
strategies to provide huge gains in productivity at modest cost. On the other 
hand, they reveal the technocratic forces that work against the application of 
such strategies. There is little doubt that the engineering staff associated with 
development of that new landmine sweeper imagined that better technology 
(twin sensing heads and improved algorithms) would resolve the problem by 
adding or enhancing critical detection functionality. Even the name given this 
device, the Handheld Standoff Mine Detection System (Cyterra, 2015), 
implies that all of the important functionality resides within the equipment. 
The role of the worker is trivialized, being consigned to one of merely 
holding or carrying the device. The work of Stazewski (2004) reveals that 
this implication is not only inaccurate but so completely distorted as to be 


naive. 


A similar techno-centric mindset was evident in the emergency response 
research reported by Klinger and Klein (1999), where the emergency 
planning director gravitated naturally to technology as a solution to his 
problem without considering the possibility that adjustments in workflow and 
work organization could produce the desired result. 

There is a further problem. It is all too common for those with the 
techno-centric mindset to default to algorithmic solutions for challenges to 
human judgment and this attitude can be found even in those whose 
professional expertise is in behavioral science (e.g. Kahneman, 2011). As 
illustrated in the landmine research, the algorithmic approach can fail to 
provide any worthwhile enhancement; Stazewski’s data show it can actually 
result in degraded performance. Furthermore, algorithms do not inevitably 
translate into better performance or a requirement for fewer workers and they 
typically create a need for new or different kinds of expertise (Bradshaw, 
Hoffman, Johnson, & Woods, 2013; Defense Science Board, 2016; Hoffman, 
Cullen, & Hawley, 2016). 

Kahneman and Klein (2009) reflected on the role for algorithms by 


asserting that “people perform significantly more poorly than algorithms in 
low-validity environments” (p. 523). More accurately referred to as a low- 
coherency environment, a low-validity environment is one in which the 
available information does not clearly specify the state of the situation. The 
detection of low-metal anti-personnel mines would, at first glance, appear to 
be such a situation. From that perspective, the development of better 
algorithms offers the most (and, some may assume, the only) promising 
approach. However, the fact that some workers can find information to 
support high-level performance establishes conclusively that this is not a low- 


coherency environment. As shown here, the all-too-ready assumption that 


conditions are such that only a technology solution will be viable is not only 
shortsighted but can also be counterproductive. 

Staszewski’s data also demonstrate the problem with the claim that is 
sometimes made that with sufficient experience and enough rapid feedback, 
humans will find the relevant information if it exists. Staszewski’s landmine 
work suggests that the development of expertise is not that straightforward. 
In some cases, a training regimen will be needed to direct the workers’ 
attention to critical information, and workers need to be motivated to engage 
in deliberate practice. Stazewski’s data further suggest that a systematic 
cognitive analysis is a prerequisite for establishing whether or not a situation 
has low information coherency and also for identifying the information that 


can support expert performance. 


Foundational Methods 


In this section, we describe three established methods from Cognitive 
Systems Engineering: the Critical Decision Method, Concept Mapping, and 
the Decision Ladder. In the previous edition of this book, we described Work 
Domain Analysis, but in this edition, we have substituted the Decision 
Ladder in preference to Work Domain Analysis because, in recent years, 
there have been some useful developments in how Decision Ladders can be 


understood and developed. 


The Critical Decision Method 


The Critical Decision Method is a semi-structured cognitive interview that 
uses probe questions to explore the cognitive dimensions of decisions made 
by experienced workers during non-routine or challenging incidents. When 
using this method, the cognitive engineer, as interviewer, elicits stories about 
non-routine events in which experienced workers used special cognitive skills 
to resolve challenging situations. The Critical Decision Method leverages the 
fact that workers often retain detailed memories of previously encountered 
cases, especially ones that were unusual or challenging. 

The stories are of lived cases that were especially challenging for the 
individual who was making the important decisions. Stories about time- 
pressured, non-routine, high-stakes events, especially those where an 
inexperienced individual might have faltered, can provide a rich source of 
cognitive elements of the sort that the Critical Decision Method is designed 
to reveal. The probe questions elicit information about how the worker sized 
up the situation and then decided to take the action that resulted in the 
outcome. 

Story retelling cycles through sweeps in which the interviewer elicits 
qualitatively different knowledge from the worker on each sweep. The first 
sweep elicits a brief version of a number of stories, allowing the interviewer 
to select one for further analysis. The second sweep allows the worker to 
relate the selected story as the interviewer drafts a timeline. The third sweep 
uses probe questions to elicit details about significant cognitive states and 
processes such as how the worker assessed the situation, what they noticed 


when reassessing the situation, what alternate goals they considered, and 


what courses of action they considered. The fourth sweep uses hypothetical 
probes, such as queries about what a worker might do if a crucial piece of 
information were unavailable or the sorts of mistakes less experienced 
workers might make. 

In summary, the Critical Decision Method elicits information within a 
specific challenging incident about cognitive functions such as decision 
making, planning, and sense-making. Experts are asked to describe decisions 
they made during an incident and also to describe the information and rules 
of thumb they used during the decision process. Experts are further asked to 
identify situational features that might have made decisions difficult and 
situational elements that characterized the incident as familiar. 

This method of knowledge elicitation is used to uncover elements of 
expert reasoning such as decision types, decision strategies, decision 
requirements, informational triggers, and hidden assumptions. The goal is to 
develop a deep appreciation of the style of cognition employed by 
experienced workers as they engage in non-routine or challenging incidents. 
Comprehensive discussions of the Critical Decision Method are presented by 
Hoffman et al. (1998) and Crandall et al. (2006). 


Concept Maps 


A Concept Map is a meaningful diagram that includes concepts (enclosed in 
boxes) and relationships among concepts (indicated by labeled connections 
between related concepts). The concepts and relationships can be read as 
meaningful propositions where propositions take the triadic form of 
[Concept] Relation [Concept]. Figure 11.3 depicts a Concept Map of the field 
of Social Network Analysis. Within this Concept Map are propositions such 


as: 
[Social Network Analysis] has [Goals] 
[Data Collection] involves [Sampling] 


A Concept Map has some features of a process description and some features 
of a hierarchy but it is not strictly either, primarily because of the conceptual 


cross-links. 
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Figure 11.3 Concept Map created in knowledge elicitation with an expert 
in Social Network Analysis. 


In contrast to the Critical Decision Method, where the focus is on 
judgments and decisions made in actual incidents, Concept Mapping is aimed 
at generating models of the abstract knowledge that supports cognition in a 
particular work domain. The method has its foundations in the theory of 
Meaningful Learning (Novak & Gowan, 1984) and decades of research and 
application in education (Novak, 1998) where Concept Maps are most often 
used to show gaps in student knowledge. Over the past couple of decades, 
Concept Maps have been used to capture expert knowledge in a wide range 
of work domains (see Hoffman, LaDue, Mogil, Roebber, & Trafton, 2017; 
Moon et al., 201 1a). 

In the knowledge elicitation procedure, an interviewer serves as the 


facilitator of the Concept Mapping Method while either that interviewer or a 


second facilitator builds the Concept Map. The map may be developed with 
pen and paper but software tools are available and offer the advantage that 
the map can be projected onto a screen in full view of the interviewer and the 
domain expert as it is being developed. The interviewer helps the domain 
expert build up a representation of their domain knowledge, in effect 
combining knowledge elicitation with knowledge representation. By 
integrating the development of the knowledge representation with the process 
of knowledge elicitation, Concept Mapping is time efficient because it 
bypasses the time-consuming effort associated with extracting knowledge 
from interview records. 

Moon, Hoffman, Eskridge, and Coffey (20116) discuss a range of skills 
that concept mappers need if they are to conduct Concept Mapping efficiently 
and effectively. These include facility with software tools, the ability to 
Manage space within the map, and skill at propositional thinking. Notably, 
with some training in the method, Concept Mapping can also be conducted 
by domain experts without the aid of an interviewer, a feature that 
distinguishes the method from most other methods of knowledge elicitation 
and representation. 

While a single Concept Map can express aspects of expertise in a 
domain, multiple Concept Maps are needed to represent an entire domain. 
The ability afforded by software tools to hyperlink and navigate many 
Concept Maps enables the creation of comprehensive models of large, 
knowledge-intensive domains. Additional digital resources such as text 
documents, images, video clips, and URLs can be linked to concepts, creating 
knowledge models that can serve as living repositories of expert knowledge 
to support knowledge sharing as well as knowledge preservation. 


In summary, a Concept Map is a graphical representation of the abstract 


knowledge that supports cognition in a particular work domain, where a 
concept is an idea generalized from a set of particular instances. As such, a 
concept represents a regularity across similar (although not identical) events 
or situations. A Concept Map organizes expert knowledge in the form of 
interlinked propositions, where a proposition takes the form of two concepts 
linked by a statement of relationship. The goal of Concept Mapping is to 
document the abstract domain knowledge of experts in a representational 
form that helps the cognitive engineer develop a deep appreciation of the 
knowledge that supports expert activity in a work domain. 

Reviews of the literature and discussion of methods for making Concept 
Maps can be found in Cafias et al. (2003) and Crandall et al. (2006). 


Work Task Analysis 


Work Task Analysis, also known as Control Task Analysis (Vicente, 1999), 


is based on the assumption that activities are undertaken, problems resolved, 
and decisions made via transformations between cognitive states as induced 
by cognitive processes. Thus, work tasks can be described in terms of the 
cognitive states established during work activity and the cognitive processes 
used to effect the transitions between states (Lintern, 2010). The usual 
product of Work Task Analysis is a suite of Decision Ladders. 

A cognitive state is a condition of being (e.g. the state of being alert, the 
state of being aware of the situation, the state of being certain or uncertain, 
the state of knowing something) while a cognitive process is an activity (e.g. 
the process of seeking information, the process of planning). In a physical 
system, a state is a condition described in terms of phase, form, composition, 
or structure (e.g. ice is the solid state of H,O and water is its liquid state). A 
physical process acts on a state to change it (e.g. the process of cooling 
transforms water into ice). There can be no state transition in a physical 
system without an intervening process. Cognitive states and processes can be 
viewed similarly, although in the realm of cognition, processes are not always 
accessible to conscious awareness, in which case they are said to be implicit. 

The product of Work Task Analysis is a Decision Ladder as illustrated 
in Figure 11.4. It provides a template for mapping the set of generic sub-tasks 
involved in decision making (Rasmussen, Pejtersen, & Goodstein, 1994, 
p. 66), that is, the cognitive states (depicted as ellipses) and the cognitive 
processes (depicted as arrows). A work narrative can be mapped onto the 


Decision Ladder to represent observed decision paths and to identify different 


decision processes (Rasmussen et al., 1994, p. 66). As depicted in Figure 
11.4, the Decision Ladder has three main stages, situation assessment (left 


hand leg), options analysis (across the top), and planning (right hand leg). 
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Figure 11.4 The Decision Ladder (adapted from Rasmussen, 1986 and 
Rasmussen et al., 1994). 


A Decision Ladder accommodates an opportunistic style of cognitive 
activity. The trajectory of a work task can start and finish anywhere in the 
ladder and can transition across the ladder, as illustrated by the formulate 
plan and the formulate and evaluate plan links in Figure 11.4. Additionally, 
processes may be either explicit or implicit. An explicit process is accessible 


to conscious awareness while an implicit one is not. In Figure 11.4, explicit 


processes are depicted by solid arrows while the illustrative implicit process, 
Perceive—Execute, is depicted by a dotted arrow. 

A work narrative, once mapped onto a Decision Ladder (as shown in 
Figure 11.4) can be read to gain insight into the cognitive states achieved and 
the cognitive processes employed during work activity. A work narrative that 
follows the perimeter of the Decision Ladder, starting at the lower-left node 


and finishing at the lower-right node might be read as follows: 


A worker who is immersed in a work situation will be aware of the types 
of events that demand intervention. On perceiving an event and 
becoming alerted to or aware of information that characterizes it, they 
will engage in diagnosis to discover what is going on. They will first 
seek information about the work demands and the surrounding 
conditions and with that information in hand, they will seek to 
comprehend the current system state in relation to current goals and to 
anticipate the future system state (given no intervention) while 
remaining cognizant of situational exigencies that may demand 
reassessment. They will then identify a desirable and reachable system 
state. Alternatively, it may be difficult to identify a desired system state 
directly from the comprehension of the future system state, in which case 
the worker will divert through the options analysis loop to identify and 
then evaluate options for desired states in order to compare the 
consequences of those options as a prelude to settling on a desirable 
system state. Once a desirable system state is identified, there will be a 
need to formulate a plan. Once that is done, the worker will evaluate the 


plan. If satisfied, they will execute the plan. 


A trajectory for knowledge-driven work as mapped onto a Decision Ladder 
should not be interpreted as implying a fixed sequence of cognitive states and 
processes for all, or even for any work activity undertaken in pursuit of 
similar goals. Work task trajectories are influenced by situational exigencies 
and the fluidity of personal preference. 

Figure 11.5 shows an anesthetist narrative mapped onto a Decision 
Ladder. This narrative, taken from the first column of the Cognition 
Requirements Table of Table 11.1, identifies the cognitive states and 
cognitive processes that are activated as an anesthetist attempts to ventilate 
and then intubate a patient with low oxygen saturations. The anesthetist is 
aware that the patient needs to be oxygenated (Figure 11.5, entries 1, 2, 3) 
and attempts to accomplish that by ventilation with a bag-mask (4). The bag- 
mask intervention fails (5, 6) and the anesthetist then attempts to insert a 
breathing tube, using a laryngoscope to guide the tube past the patient’s 
pharynx (7). This also fails because, as becomes evident during the attempted 
intubation, the patient’s airway is so swollen that the anatomical landmarks 
normally used to guide tube insertion are not visible (8). The anesthetist 
declares failed intubation and failed ventilation (9) and decides to open the 
airway surgically with a needle (10). As evident in Figure 11.5, many of the 
cognitive processes used to transition from one cognitive state to another are 


implicit (indicated by dotted versus solid arrows). 
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Figure 11.5 The cognitive states and processes (with sequence indicated 
by numbers) activated as an anesthetist attempts to ventilate and then 
intubate a patient with low oxygen saturations (condensed from a transcript 
provided by Schnittker, Marshall, Horberry, Young, & Lintern, 2016). 


In execution of a work task, different workers at different levels of 
experience may follow different trajectories, and the same worker may 
opportunistically follow different trajectories at different times for the same 
work. An expert is likely to visit fewer cognitive states and to employ fewer 
cognitive processes than a novice, but might also choose different trajectories 
at different times. That is not to say that anything is possible; the chosen 
trajectory must reflect the needs of the work. Why a worker (whether novice, 
apprentice, journeyman, or expert) might choose a different trajectory at a 
different time will often be an interesting issue to explore, one that might 


have ramifications for the design of a cognitive support tool. 


Enhanced cognitive support might come through one (or some 
combination) of technological redesign, work process redesign, or training, 
focused on the specific cognitive states or processes (or some combination of 
them) that offer a challenge in execution of work. Every cognitive state and 
every Cognitive process involved in work activity is a candidate for assistance 
with some form of technological, process, or training support. Figure 11.6 
offers a sample of potential design interventions that could support work 
activity. Whether any form of support is desirable for any specific (or 
combination of) cognitive states or processes will depend largely on whether 
those states or processes (combined or in isolation) offer particular cognitive 


challenges that could be eased by the form of support being proposed. 
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Figure 11.6 A sample of potential design interventions. 


As shown in Table 11.1, the development of a Cognition Requirements 
Table for the anesthetist scenario described above suggests development of 
enhanced technological support to assist intubation and also a greater focus in 
training on difficult cases. Similar conclusions might be drawn by inspection 
of the Decision Ladder of Figure 11.5, although the particular design 
solutions that would emerge from either approach depend largely on the 
knowledge and creativity of the design team. 

In summary, Work Task Analysis assumes that work is accomplished, 
problems resolved, and decisions made via transformations between 
cognitive states as induced by cognitive processes. Work Task Analysis 
identifies the cognitive states and cognitive processes used within a work task 
by mapping trajectories of work narratives as provided by expert or 


competent professionals onto a Decision Ladder. 


Summary: Foundational Methods 


In eliciting expert knowledge, researchers can create models of the work 
domain, of practitioner knowledge, and of practitioner decision making. The 
Critical Decision Method can be used to create products (e.g. the Cognitive 
Requirements Table) that describe practitioner reasoning (e.g. decision types, 
strategies, decision requirements, information). Concept Mapping represents 
practitioner knowledge of concepts and their relations in the context of a 
worker’s understanding of a domain. Work Task Analysis maps the trajectory 
of cognitive states and cognitive processes activated within work tasks onto 
Decision Ladders. Each of these methods has proven useful for capturing and 
representing detailed expressions of cognitive work as needed to inform 


Cognitive System Engineering design efforts. 


Emerging Methods 


In this section, we describe two emerging methods that offer additional tools 
for exploring cognitive work. Macrocognitive Modeling is a new and more 
comprehensive method for capturing and representing expertise while 
ShadowBox is a new method for injecting subtle aspects of expertise into 


cognitive skills training. 


Macrocognitive Modeling 


The word “macrocognitive” is a reference to a new designation for the 
paradigm of Naturalistic Decision Making. That paradigm gained traction in 
part from the results of groundbreaking studies of experts, such the firefighter 
studies of Klein (1989) that led to development of the Recognition Primed 
Decision Model. This new designation of “macrocognition” emerged out of 
recognition that naturalistic inquiry into cognitive work is about more than 
just decisions; it also references other high-level cognitive processes such as 
sense-making, re-planning and coordinating (Klein & Hoffman, 2008). (For 
more background on Macrocognition, see Mosier, Fischer, Hoffman, & 
Klein, Chapter 25, this volume.) 

Macrocognitive Modeling was developed to generate and validate 
macrocognitive models of practitioner reasoning. It evolved after Hoffman 
and colleagues (2017) had developed a general model of expert weather 
forecaster reasoning that integrated classic models of problem-solving (e.g. 
Duncker, 1945; Newell, 1985) with models of expert reasoning (e.g. Kruger 
& Cross, 2006; Kuipers & Kassirer, 1987; Rasmussen, 1986). 


Macrocognitive models assume that high-level cognitive processes are 
parallel and highly interactive. They differ fundamentally from more 
traditional models that describe cognition in terms of steps or stages, such as 
input-output models. All such models are based on causal chains and focus 
on micro-scale cognitive processes (such as short-term memory access). 
Macrocognition assumes a dynamic, ongoing, and iterative engagement with 
the essential cognitive work. Thus, Macrocognitive models contain a number 


of closed loops, some of which are embedded in others. The model shown in 


Figure 11.7 captures a considerable number of hypothetical reasoning 
sequences that have been proposed in studies of expertise and problem- 


solving (see Hoffman & Militello, 2008). The embedded closed loops are 


indicated by the arrows. 
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Figure 11.7 The “base model” of expert reasoning (from Hoffman et al., 


2017). 


A version of this model was next created to capture results from studies 
of weather forecaster reasoning. This is shown in Figure 11.8. The 
Macrocognitive Modeling procedure as described here was first conceived 
when a weather forecaster was shown the Figure 11.8 model and was asked 
whether it seemed to describe their work process. That forecaster 


spontaneously added domain-specific details to the process description and 


modified some of the relations among the diagram elements. This experience 
prompted the research team to develop a more systematic knowledge 


elicitation procedure to capture expert reasoning (rather than expert 
knowledge). 
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Figure 11.8 A Macrocognitive Model of expert forecaster reasoning (from 
Hoffman et al., 2017). 


That more formal procedure consisted of three basic steps: 


Step 1: Preparation. The base model (Figure 11.7) was adapted to make 
it directly pertinent to the domain. Two alternative “bogus models” that 


included some of the elements of the base model were then created. 


Step 2: Model Making. Forecasters spanning a range of proficiency 
were shown the alternative models and invited to select the one that best 
represented their forecasting strategy. Then, using that selected model as 
a scaffold, forecasters were invited to concoct their own reasoning 


diagram. 


Step 3: Verification. After a few days or more, the researcher observed 
forecasters as they began work. Some elements of the model were 
validated by observation (e.g. examine satellite images), whereas other 
elements were subjected to probe questions (e.g. what are you thinking 
now? what are you doing now?). Figure 11.8 shows results for just one 
of the forecaster models. The qualifications express the results from the 
verification process, that is, an activity could be affirmed or disaffirmed 


by observation or by probe question response, or otherwise qualified. 
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Figure 11.9 Results from an observational verification of a forecaster’s 
model (from Hoffman et al., 2017). 


This first use of the Macrocognitive Modeling procedure produced 
reasoning models for seven proficient forecasters (both journeymen and 
experts). The results validated the general model of forecaster reasoning 
developed from an initial document analysis (Figure 11.8) and revealed clear 
differences in proficiency among the group of forecasters. Less capable 
forecasters relied uncritically on computer forecasts and were less likely to 
think hypothetically and counterfactually. They were also more likely to rely 


on fixed sequences for inspecting outputs of various computer models of the 


weather. More generally, this method helps a cognitive engineer develop 
insight into how workers at different levels of expertise think about their 


work. 


ShadowBox™ 


ShadowBox represents a recent advance in accelerating the cognitive 
competency of workers toward high levels of proficiency. It accelerates 
progression toward expertise by using elicited knowledge in the development 
of a training system. ShadowBox was originally developed by Hintze (2008) 
and further elaborated by Klein, Hintze, and Saab (2013). 

The training regime uses a challenging scenario with several decision 
points interspersed through it. Each decision point has a small set of options 
defined by different courses of action, different goals, different situational 
interpretations, or different types of information. A panel of operational 
experts works independently through the scenario to rank the options and 
describe the rationale for their choices. Each trainee follows a similar 
procedure, rank-ordering the options and providing a rationale for their 
rankings. The trainee then reviews the rankings and rationale provided by the 
operational experts. In this way, the trainee has the chance to see the world 
(the scenario) through the eyes of the experts without having the experts 
present. 


Several training studies (Klein & Borders, 2016) have shown that 


ShadowBox can, with relatively little training (a half-day or less), increase 
the correspondence between trainee and expert rankings by 20 percent to 28 
percent. While ShadowBox has primarily been used in training, it can also 
support elicitation and representation of expert knowledge within the context 


of the scenarios. 


Application Areas 


Cognitive Systems Engineering gained momentum in the early 1980s as a 
subdiscipline of Human Factors. From the outset, the focus of concern was 
with the way in which experienced workers at a reasonable level of expertise 
performed their work. It quickly became apparent that many of the 
hypothetical constructs favored by theoretically oriented cognitive 
psychologists did not offer much in the way of explanatory value for how 
work was actually accomplished. That observation motivated an interest in 
natural work settings. In those early days of Cognitive Systems Engineering, 
a good deal of knowledge elicitation and knowledge representation was 
undertaken with electronic troubleshooters (Rasmussen, 1986), fire 
commanders (Klein & Calderwood, 1991), nuclear power operators (Woods 
& Roth, 1986), and neonatal intensive care nurses (Crandall & Getchell- 
Reiter, 1993). This work served to seed the field and develop a momentum 
such that cognitive engineers as members of the Cognitive Engineering and 
Decision Making Technical Group now make up the largest technical group 
within the Human Factors and Ergonomics Society. 

Healthcare has emerged as an important application area for Cognitive 
Systems Engineering. The Healthcare Technical Group is currently the 
second largest technical group within the Human Factors and Ergonomics 
Society. Since 2012, this Technical Group has run an annual healthcare 
symposium separate from the regular annual symposium. Approximately half 


of the papers in that symposium addressed cognitive issues, with many of 


those papers employing knowledge elicitation and knowledge representation 
strategies similar to those we have outlined in this chapter. 

Research by Moon and colleagues (Moon, Hoffman, Lacroix, Fry, & 
Miller, 2014) with healthcare professionals is illustrative. Moon et al. adapted 
the Critical Decision Method to the healthcare context and to interviewee 
proficiency levels to elicit details about cognitive and collaborative work, 
technologies and tools, and difficult cases. They focused on the central issues 
of decision making (Hoffman & Yates, 2005) and improvisations inherent in 
working within socio-technical work systems (Hoffman, Norman, & 
Vagners, 2009). The knowledge gathered by this effort, represented in 
extensive decision requirements tables and concept maps, informed the 
development and validation of guidelines for clinical decision support 
through iterative development and formative evaluation of prototype user- 
interfaces (Moon, Way, Fry, Miller, & Montella, 2015). 

Methods of knowledge elicitation and representation have been applied 
to an emerging concern with managing and controlling autonomous air 
vehicles (Dominguez, Strouse, Papautsky, & Moon, 2015; Papautsky, 
Dominguez, Strouse, & Moon, 2015). In this work, a team of cognitive 
engineers was charged with designing two interfaces for control of a full- 
sized autonomous helicopter. One interface was for a handheld device to 
support non-aviator warfighters at remote combat outposts as they placed 
resupply orders, monitored missions, and interacted with the unmanned 
helicopter to approve its landing, to offload it, and to send it on its way. The 
other interface was for ground-control station at the main operating base to 
support fully trained operators of unmanned aerial systems in their planning 
and execution of missions. 


The team developed task diagrams (Hutton & Militello, 1996) to sketch 


the timeline for resupply missions. Tasks and events known to occur along 
the timeline were identified to establish a structure for organizing the elicited 
knowledge. Subsequently, an explicit representation of the unfolding mission 
timeline became an interface requirement, with the handheld device operator 
receiving an abstracted view and the ground-control station operator 


receiving a detailed view. 


Conclusion 


The discipline of Cognitive Systems Engineering has now existed for 
approximately four decades. During that time, cognitive engineers have 
continued to develop its methods of knowledge elicitation and knowledge 
representation. Since the first edition of this handbook there have been 
significant developments in the methodology and applications of knowledge 
elicitation and knowledge representation as undertaken within natural work 
settings. Furthermore, cognitive engineers have continued to refine not only 
the methods themselves but also the way those methods are explained. Our 
effort in the first edition of this handbook offered some advance on the 
explanations and descriptions available at that time and we hope that our 
effort for this edition of the handbook offers a further advance. 

Design sensitivity to expert knowledge as promoted by the methods of 
knowledge elicitation and representation we describe here is sorely needed. 
Despite the techno-centric design failures as described by Challenger et al. 
(2013) and Moon and Hoffman (2005), and the cognitive engineering 
successes as described by Staszewski (2004) and Klinger and Klein (1999; 
see also Cooke & Durso, 2007), acceptance of the sorts of methods we 
describe here is narrow. Where human systems challenges are concerned, the 
tendency is to turn to technological solutions, and even worse, solutions that 
in large part ignore the challenges, the practices, and the hard-won expertise 
of those who must perform the work. In healthcare, for example, far-reaching 
decisions about the development of information and coordination systems are 


made collaboratively where committee members, although often healthcare 


professionals, have specific and narrow (albeit deep) expertise that does not 
aggregate into the total of the systems-wide expertise that supports 
information flows and collaborations within a large-scale, distributed 
healthcare system. 

There is enormous potential to enhance the effectiveness of cognitive 
work systems by understanding how experts go about their work. There is 
also enormous potential to enhance system performance by using expert 
knowledge as the basis for design of technologies that can support human 
activity over the full range of work situations and full range of work 
capability from novice to expert. In addition, there is also enormous potential 
to enhance system performance by using that knowledge as the basis for 
training to facilitate the progress of apprentices, novices, and journeymen 
along the path to expertise far more quickly than they ever could by 
themselves (Hoffman et al., 2014). The methods and concepts outlined in this 


chapter show how those benefits might be realized. 
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The superior performance of experts, such as accomplished musicians and 
chess masters, can be amazing to most spectators. For example, club-level 
chess players can be surprised by the moves selected by grand masters until 
they can see the consequences of the moves on the resulting chess position 
after several subsequent moves. Similarly, many recreational athletes find it 
inconceivable that most other adults — regardless of the amount or type of 
training — have the potential ever to reach the performance levels of 


international competitors. Philosophers and scientists have argued for 


centuries about the relative importance of innate gifts versus specialized 
acquired skills and abilities for attaining expert performance. 

One of the most widely used and most direct methods for gathering 
information on exceptional performance is to interview the experts about how 
they think that they are different and how the development of their superior 
performance differs from other people and less accomplished adults. Among 
scientists there is controversy as to whether experts have knowledge about 
these differences and if they are capable of describing them in an accurate 
and valid manner. Without objective evidence on how experts are different, 
researchers have found evidence raising doubts about the validity of the 
experts’ introspective verbal reports. 

Some pioneering scientists studying experts, such as Binet (1966), 
questioned the validity of the experts’ descriptions when they found that 
some experts gave reports inconsistent with those of other experts. One 
central difficulty with self-observations (introspections) is that you need to 
trust the person reporting them to give these reports accurately. If different 
experts give different reports then there is no objective way to resolve any 
controversy between the experts. In a few rare cases scientists have been able 
to observe the experts’ performance and where they are directing their gaze, 
then compare those observations to the experts’ verbal descriptions of what 
they perceived during the performance. In a study of golf players, Watson 
(1913) found discrepancies between the strategies reported by the experts and 
the objective observations of their actual behavior. Some of these 
discrepancies were explained, in part, by the hypothesis that some processes 
were not normally mediated by awareness/attention and that the act of self- 
observation (introspection), typically generated after the completed 


performance, did not reflect the thought processes that generated the original 


performance. These problems led most psychologists in the first half of the 
twentieth century to reject all types of introspective verbal reports as not 
being valid scientific evidence (Boring, 1950). 

Instead of collecting introspective analyses of images and thoughts and 
treating this type of data as privileged evidence, investigators such as John B. 
Watson (1920) and Karl Duncker (1945) introduced a new type of method to 
elicit verbal reports of thinking. The subjects were asked to “think aloud” and 
give immediate verbal expression to their thoughts while they were engaged 
in problem-solving. This chapter will discuss how this type of verbal 
expression of thoughts can be viewed as behavior that can be elicited without 
changing the underlying structure of the thought processes and thus avoids 
the problem of reactivity induced by the process of analysis and 
introspection. In particular, I will describe the methods of protocol analysis 
where verbal reports are elicited, recorded, and encoded to yield valid data on 
the underlying thought processes (Ericsson & Simon 1980, 1984, 1993), 
Most importantly, I will describe how these data can be converted into 
testable hypotheses that can be evaluated by testing performance under 
designed experimental conditions. 

This chapter describes the general methods for eliciting concurrent and 
retrospective verbal reports for performance on tasks and how it is possible to 
analyze the reported thoughts to develop hypotheses about the mediating 
cognitive processes that can be tested by experiments. These methods play a 
central role in the expert performance approach (Ericsson & Smith, 1991; 
Ericsson & Ward, 2007; Ericsson & Williams, 2007), where superior 
performance of experts can be captured and reproduced in the laboratory to 
assess the mediating cognitive processes. The application of these methods to 


a number of domains of expertise, such as memory experts, chess masters, 


and medical experts, will be described. The chapter concludes with a 
discussion of issues raised by applying protocol analysis to the study of 


expert performance. 


Brief History of Verbal Reports on Expert 
Thought Processes 


Most of the introspective analyses of thinking were based on self-analysis of 
individual philosophers’ own thoughts. In the nineteenth century Sir Francis 
Galton along with others introduced several important innovations that set the 


groundwork for empirical studies of thinking. For example, Galton (1879; see 


Crovitz, 1970) repeatedly noticed that when he took the same walk through a 


part of London and looked at a given building on his path this event 
frequently triggered the same or similar thoughts retrieved from memory. 
Galton recreated this phenomenon by listing the names of the major buildings 
from his walk on cards and then presented one card at a time to himself and 
recorded the thoughts that were triggered. From this self-experiment Galton 
argued that thoughts reoccur frequently when the same stimulus (card with a 
particular name) is encountered. 

In one of the first published studies on expertise Binet (1966) reported 
on pioneering interviews of chess players with a focus on their ability to play 
“blindfolded” without seeing a chess board. Based on his interviews Binet 
concluded that the ability required to maintain chess position in memory 
during blindfold play did not appear to reflect a superior basic memory 
capacity to store complex visual images, but rather depended on a deeper 
understanding of the structure of chess positions. Binet found dramatic 
individual differences in the images reported by the chess players to mediate 
their blindfold chess playing. Binet (1966) was unable to determine whether 


the individual differences in the reported visual acuity of their imagery were 


valid or corresponded to problems with the reporting method. The only way 
to go beyond these descriptions would require an extended analysis of the 
blindfold performance of individual chess players and the collection of 
concurrent reports on their thoughts to assess if reported differences between 
experts can be confirmed. 

In a similar manner Bryan and Harter (1899) interviewed two students 
of telegraphy as they improved their skill and reached an extended 
performance plateau, which happened when both of them reached a rate of 
receiving messages at around 12 words per minute. Both reported that this 
arrest in development was associated with attempts to move beyond encoding 
the Morse code into word units and start encoding the code into word 
sequences and phrases. Decades later Keller (1958) studied this phenomenon 
with experimental methods. He was able to show that the plateaus reported 
by Bryan and Harter (1899) were a consequence of the strategies used by 
their participants. By instructing participants to use different strategies Keller 
(1958) showed that these plateaus are not a necessary intermediate step 
toward expert levels of performance and referred to them as examples of “the 
phantom plateau.” 

Many investigators, including the famous behaviorist and critic of 
analytic introspection, John B. Watson (1913), were very critical of the 
accuracy of introspective descriptions. In a subsequent paper Watson (1920) 
offered an alternative to introspective analysis and was the first investigator 
to publish a study where a subject was asked to “think aloud” while solving a 
problem. According to Watson, thinking was accompanied by covert neural 
activity of the speech apparatus that is frequently referred to as “inner 
speech.” Hence, thinking aloud was merely giving overt expression to sub- 


vocal verbalizations by vocalizing them aloud. This process is quite different 


from the traditional introspection, which often referred to mental imagery as 
being projected inside the head and then having a “man in the head” that 
observed and reported on the images. This does not really explain how 
introspections are generated because there is no specification of the processes 
of the “man in the head,” because that would require a smaller man inside the 
head of the “man in the head” and so on. To avoid such problematic 
introspective mechanisms other contemporary investigators proposed similar 
types of instructions to the vocalization of inner speech where participants 
were asked to give concurrent verbal expression of their thoughts while 
performing a given task (see Ericsson & Simon, 1993, for a more extended 
historical review). 

The emergence of computers in the 1950s and 1960s and the design of 
computer programs that could perform challenging cognitive tasks brought 
renewed interest in human cognition and higher-level cognitive processes. 
Investigators started studying how people solve problems and attempted to 
describe and infer the thought processes that mediate successful performance. 
They proposed cognitive theories where strategies, concepts, and rules were 
central to human learning and problem-solving (Miller, Galanter, & Pribram, 
1960). Based on information processing theories, Newell and Simon (1972) 
generated designs of computational models that could regenerate human 
performance on well-defined tasks by the application of explicit 
computational procedures. Much of the evidence for these complex rules and 
thoughts was derived from the researchers’ own self-observations, informal 
interviews, and systematic questioning of research participants performing 
the associated tasks. Many of these modeling projects attempted to build 
functioning computer programs based on verbal descriptions by experts and 


participants. However, instructing individuals to think aloud while 


performing a task frequently did not produce verbal reports with sufficient 
detail about the mediating cognitive processes to permit the building of 
complete computer models. Hence, some investigators continued to search 
for alternative instructions to generate more detailed verbal reports on how 
the participants generated solutions to the problems. Frequently investigators 
required participants to explain their methods for solving tasks. These types 
of alternative reporting methods elicit additional and more detailed 
information than is spontaneously verbalized during “think aloud.” 

Some investigators raised concerns almost immediately about the 
validity of the generated explanations. For example, Robert Gagné and his 


colleagues (Gagné & Smith, 1962) demonstrated that requiring participants to 


verbalize reasons for each move in the Tower of Hanoi improved 
performance by reducing the number of moves in the solutions and improved 
transfer to more difficult problems when compared to a silent control 
condition. Although improvements of cognitive performance are welcomed 
by educators, any improvement due to explaining must have changed the 
sequences of thoughts compared to the thought sequences that are normally 
generated while performing at a lower level. 

Other investigators criticized the validity and accuracy of the 
retrospective verbal explanations. For instance, Nisbett and Wilson (1977) 
reported several examples of experiments in social psychology, where 
participants subsequently were asked to explain the reasons for a decision. 
The participants’ explanations of their decisions were indistinguishable from 
explanations generated by people who only observed these participants’ 
behavior. Nisbett and Wilson (1977) proposed that even the participants 
inferred their reasons from their own observable behavior and that they 


lacked any privileged information about their decision making. These 


findings initially led many investigators to conclude that all types of verbal 
reports were tainted by similar methodological problems that had plagued 
introspection and led to its demise. Herbert Simon and I showed in a review 
(Ericsson & Simon, 1980) that the methods and instructions used to elicit the 
verbal reports had a great influence on both the reactivity of the verbal 
reporting and the accuracy of the reported information. We developed a 
particular methodology to instruct participants to elicit consistently valid and 


non-reactive reports of their thoughts that is described in the next section. 


Protocol Analysis: A Methodology for 
Eliciting Valid Data on Thinking 


The central assumption of protocol analysis is that it is possible to instruct 
subjects to verbalize their thoughts in a manner that does not alter the 
sequence and content of thoughts mediating the completion of a task and 
therefore the verbalized information should reflect immediately available 


information during silent thinking. 


Elicitation of Non-Reactive Verbal Reports of Thinking 


Based on their theoretical analysis, Ericsson and Simon (1993) argued that 
the closest connection between actual thoughts and verbal reports is found 
when people verbalize thoughts that are spontaneously attended during task 


completion. Figure 12.1 illustrates how most thoughts are given a verbal 


Thought ,| @™D |Thought ,| eee A ™® |Thought . 


Verbal Verbal Verbal 
encoding , encoding , encoding ,, 


Vocalization Vocalization , Vocalization “ 


expression. 


Figure 12.1 An illustration of the overt verbalizations of most thoughts 
passing through attention while a person thinks aloud during the 


performance of a task. 


When people are asked to think aloud (see Fox, Ericsson, & Best, 
201\1a, for complete instructions), some of their verbalizations seem to 
correspond to merely vocalizing “inner speech,” which would otherwise have 
remained inaudible. Ericsson and Simon (1980, 1993) referred to these types 
of verbalizations as Level-1. When the thoughts were not in verbal form an 
additional step was required to give these thoughts verbal expression by brief 
labels and referents, which were referred to as Level-2 verbalizations 
(Ericsson & Simon, 1980, 1993). Ericsson and Simon also distinguished 


additional processes, where participants described their thoughts and even 


tried to explain why a given thought had been generated and referred to this 
type of verbalization as Level-3. 

Laboratory tasks studied by early cognitive scientists focused on how 
individuals applied knowledge and procedures to novel problems, such as a 


mental multiplication of numbers (Dansereau & Gregg, 1966), Tower of 


Hanoi (Anzai & Simon, 1979), and physics problems from introductory 


college courses (Larkin, McDermott, Simon, & Simon, 1980). When, for 
example, one participant was asked to think aloud while mentally multiplying 
36*24 on two test-occasions one week apart the following protocols were 


recorded: 


OK, 36 times 24, um, 4 times 6 is 24, 4, carry the 2, 4 times 3 is 12, 14, 
144, 0, 2 times 6 is 12, 2, carry the 1, 2 times 3 is 6, 7, 720, 720, 144 
plus 720, so it would be 4, 6, 864. 

36 times 24, 4, carry the — no wait, 4, carry the 2, 14, 144, 0, 36 times 
2 is, 12, 6, 72, 720 plus 144, 4, uh, uh, 6, 8, uh, 864 


In these two examples, the reported thoughts are not analyzed into their 
perceptual or imagery components, but are merely vocalized as inner speech 
and verbal expressions of intermediate steps, such as “carry the 1,” “36,” and 
“144 plus 720.” Furthermore, participants were not asked to describe or 
explain how they solve these problems and their think-aloud protocols do not 
generally contain such descriptions or explanations. Instead, they are asked to 
stay focused on generating the requested response to the presented task and 
thus only give verbal expression to those thoughts that spontaneously emerge 
in attention during the generation of the response. 

If the act of verbalizing participants’ thought processes does not change 


the sequence of thoughts, then participants’ task performance should not 


change as a result of thinking aloud. In a comprehensive meta-analysis Fox et 
al. (2011a) found no evidence that the sequences of thoughts (accuracy of 
performance) changed when individuals were instructed to think aloud. The 
reviewed studies compared performance of participants thinking aloud to 
other individuals who completed the same tasks silently under the same 
experimental conditions. Participants who think aloud have been found to 
take somewhat longer to complete the tasks — presumably due to the 
additional time required to produce the overt verbalization of the thoughts. 
The same theoretical framework can also explain why other types of 
verbal reporting procedures consistently change cognitive processes. For 
example, when subjects explain why they are selecting one of several 


available actions (Gagné & Smith, 1962), they do not merely verbalize each 


thought as it emerges, they have to engage in additional cognitive processes 
to generate the thoughts corresponding to the required explanations and 
descriptions (cf. Level-3 verbalizations described by Ericsson & Simon, 
1980, 1993). Instructions to explain the reasons for one’s problem-solving 
and to describe the content of thought are reliably associated with changes in 
the accuracy of observed performance (Fox et al., 20i1la). The effects of 
verbal overshadowing (Meissner & Brigham, 2001; Schooler & Engstler- 
Schooler, 1990) are consistent with reactive consequences of enforced 
generation of extensive verbal descriptions of brief experiences (Ericsson, 
2002). Even instructions to generate self-explanations have been found to 
change (actually, improve) participants’ comprehension, memory, and 
learning compared to merely thinking aloud during these activities (Ericsson, 
1988, 2003; Neuman & Schwarz, 1998). 

In summary, adults must already possess the necessary skills for 


verbalizing their thoughts concurrently, because they are able to think aloud 


without any systematic changes to their thought process after a_ brief 
instruction and familiarization in giving verbal reports (see Fox, Ericsson, & 
Best, 2011b, for instructions, warm-up tasks, and prompts to continue 
verbalizing). 


Validity of Verbalized Information while Thinking Aloud 


The main purpose of instructing participants to verbalize their thoughts is to 
gain additional information beyond what is available with more traditional 
measures of performance. If, on the other hand, verbalized thoughts are the 
primary, or even only observable data on some specific information about 


thoughts, how can the validity of the observed information be determined? 


Theories of human cognition (Anderson, 1983; Newell, 1990; Newell & 
Simon, 1972) propose computational models to account for task performance 
by reproducing the observable aspects of the processes producing correct 
responses on well-defined tasks. One of the principal methods of generating 


models for task performance involves task analysis (Schraagen, 2006). Task 


analysis specifies a range of alternative sequences of processes that 
participants could rely on in order to generate correct answers to a task, based 
on the constraints of their prior knowledge of relevant facts and procedures. 
For example, when presented with a task involving mental multiplication 
most adults have only limited mathematical knowledge. They know the 
multiplication table and only the standard “pencil and paper” procedure 
taught in school for solving multiplication problems. Under those 
assumptions one can predict that they will solve a specific mental 
multiplication such as 36*24 by first calculating 4*36 = 144 then adding 
20*36 = 720. More sophisticated adults may recognize that 24*36 can be 
transformed into (30+6)*(30-6) and that the formula (atb)*(a—b) = a*-b? can 
be used to calculate 36*24 as 30°67 = 900-36 = 864. 

The sequence of thoughts verbalized while multiplying 24*36 mentally 


(see the protocol examples above in this section) is consistent with a 


sequence of intermediate thoughts specified by one, and only one, of the two 
possible strategies for calculating the answer. Even if a highly skilled 
participant’s think-aloud report in the multiplication task only verbalized 
“144” and “720,” the reported intermediate sums would still be sufficient to 
reject many alternative strategies for generating the answer. The most 
compelling evidence for the validity of the verbalized thoughts is available 
when the series of thoughts associated with alternative methods have been 
completed a priori. When the verbalized intermediate sums match one of 
those predicted alternative sequences of intermediate products generated by 
the task analysis, the match between hypothesized states and observed states 
provides evidence for the validity of the verbalized information. A complete 
theoretical account needs to account for the match between intermediate 
products proposed a priori by the task analysis and observed verbalized 
products of the protocol. The most parsimonious account is that the 
participant generated the answer by processes generating the intermediate 
products consistent with those proposed by the task analysis. This is 
particularly evident when the verbalizations of the intermediate sums precede 
the verbalization of the answer. 

Furthermore, verbalized thoughts are only one indicator of the thought 
processes that can be observed during the generation of the solution to a 
problem. Other indicators include reaction times (RTs), error rates, patterns 
of brain activation, and sequences of eye fixations. Given that each kind of 
empirical indicator can be separately recorded and analyzed, it is possible to 
examine the convergent validity established by independent analyses of 
different types of data. In their review, Ericsson and Simon (1993) found that 
longer RTs were associated with a longer sequence of intermediate reported 


thoughts. In addition, analyses show a close correspondence between 


participants’ verbalized thoughts and the information fixated during 
sequences of eye-movements (see Ericsson & Simon, 1993, for a review). 
Finally, the validity of concurrent, think-aloud verbalizations is most 
compelling because the thoughts are verbalized while the solution is still 
being generated but still match those intermediate products generated by the a 
priori task analysis. For tasks with relatively short response latencies (less 
than 5-10 seconds), participants are typically able to recall their sequences of 
thoughts immediately after the completion of the task and report the thoughts 
in sequence starting with the first thought that they can recall. When 
retrospective reports are given immediately after the completion of the task 
the validity of this type of retrospective report remains very high. However, 
for the completion of tasks requiring longer durations (longer than 10—30 
seconds), recall of past specific thought sequences becomes more difficult, 
and people are increasingly tempted to infer what they must have thought 


thus creating inferred reconstructions rather than reported memories. 


Protocol Analysis and the Expert 
Performance Approach 


The expert performance approach to the study of expertise (Ericsson, 1996; 
Ericsson & Smith, 1991; Ericsson & Ward, 2007; Ericsson & Williams, 


2007) searches for reproducibly superior objective performance for 


representative tasks in a domain of expertise. The core idea of the expert 
performance approach is to identify task activities that capture the essence of 
superior performance in the associated domain. The application of this 
approach involves the execution of three steps. During the first step the 
identified task activities are elicited by standardized tasks that permit the 
recreation of the superior performance in the laboratory and these tasks are 
typically well-defined and require immediate action. These tasks can then be 
presented to individuals at all levels of skill, ranging from novice to 
international level expert. During the second step participants’ thought 
processes during performance are assessed with concurrent or retrospective 
reports, reaction times, and observed eye fixations. The collected process data 
are then analyzed and hypotheses are generated for the mechanisms 
mediating the superior performance. These generated hypotheses along with 
alternative conceivable hypotheses are then tested in designed experiments. 
During the third step information is collected on the development of the 
identified mechanisms that mediate increases in performance throughout the 
individuals’ engagement in the domain. This last step will be discussed in a 


different chapter (Ericsson, Chapter 38, this volume). 


The Original Research on Expert Memory 
Performance and Its Generalization 


The precursor to the expert performance approach was conducted on superior 


memory performance (Chase & Ericsson, 1981, 1982; Ericsson, Chase, & 


Faloon, 1980) and was only later generalized as an approach to the study of 


expert performance in general. In the memory research there was no need to 
capture the reproducibly superior performance because the digit-span task 
was already a standardized task administered in the laboratory. The unique 


aspect of this research was that Chase and Ericsson (1981, 1982) collected 


retrospective reports on the encoding and retrieval processes. The third step 
concerning the acquisition of the superior memory performance was simply 
that the original memory research tested individuals, who at the start of 
training displayed the level of performance of a beginner, and only with 
practice reached an exceptional level. Changes in encoding processes were 
monitored throughout training to show the step-by-step development of the 
mediating mechanisms. More recent studies of exceptional memory have 
examined individuals who displayed an exceptional level of memory 
performance at the time of the first test (Ericsson, Delaney, Weaver, & 
Mahadevan, 2004; Ericsson & Polson, 1988; Ericsson et al., 2017; Hu & 


Ericsson, 2012). These studies provide a nice transition to research studying 


the structure of expert performance in individuals who had already attained 


an expert level of performance. 


Capturing Cognitive Processes Mediating Superior Digit-Span 
Performance 
The measurement of individuals’ digit-span involves presenting lists of digits 
by reading them at one digit per second, and then asking for immediate recall. 
The purpose is to determine the length of sequences of digits that an 
individual can recall perfectly on 50 percent of the trials. The original 
participant (SF), in the first study involving extended training on the digit- 
span, had an initial span of 7 digits and attained a span of over 80 digits after 


several hundred hours of practice (Ericsson et al., 1980). On most memory 


trials SF gave a retrospective report on his thought processes before getting 
feedback on the accuracy of his recall. When memory testing began, SF 
reported relying on forming groups of digits and rehearsing the groups prior 


to recall (Chase & Ericsson, 1981). After four testing sessions SF started to 


focus on the first three digits to encode and store them in long-term memory 
(LTM), before he turned his attention to rehearsing the last group of 4-6 
digits. SF reported associating the 3-digit group with running times for 
different race distances — SF was a cross-country runner. Eventually he 
would encode a series of 3- and 4-digit groups before turning to rehearse the 
last digits of a presented list. Rather than accepting SF’s reports at face value, 


Bill Chase and I (Chase & Ericsson, 1981) designed experiments to test the 


role of mnemonic encoding, by presenting 3-digit groups that could be 
encoded as running times, such as 406 as 4 minutes and 6 seconds, and 3- 
digit groups that could not be encoded as times, such as 397. The experiment 
confirmed the essential role of mnemonic encoding. We also tested SF’s 


LTM encoding of sequences of 3-digit groups by cued recall and found 


minimal effects of concurrent chanting, which was presumed to interfere with 
phonemic rehearsal. Based on the results of over a dozen experimental 
studies, the hypotheses of the mediating mechanisms inferred from SF’s 
verbal reports were consistently supported. When SF was given transfer tasks 
involving memorizing matrices of digits, his verbal reports provided 
compelling evidence of SF’s ability to use the same encoding mechanisms for 
this type of material (Ericsson & Chase, 1982). 

A similar approach was applied to an individual (Rajan) with 
exceptional memory and a superior digit-span of over 50 digits. Rajan’s 
performance was hypothesized to reflect, in part, a superior memory capacity 
for symbols (Thompson, Cowan, & Frieman, 1993). In a series of 
experiments Ericsson et al. (2004) found that Rajan’s verbal reports while 
memorizing lists of non-numeric symbols, reflected mnemonic encoding of 
groups of symbols. Ericsson et al. (2004) were able to account for Rajan’s 
superior memory without the need for assuming innate superior memory 
capacity. Other studies have used the expert performance approach with 
protocol analysis and designed experiments to identify the mediating 
acquired mechanisms. These studies have accounted for a_ waiter’s 
exceptional memory for dinner orders (Ericsson & Polson, 1988), and a 
person who memorized over 60,000 decimals of the mathematical constant pi 
(Hu & Ericsson, 2012). 


The Expert Performance Approach with 
Protocol Analysis and Designed Experiments 


It is possible to measure someone’s chess rating by collecting data from 
many games in tournaments and to obtain a reliable measure that would 
correspond to several hundred hours of chess playing. Similarly it is possible 
to measure mortality and rates of complications for surgeons to identify 
meaningful differences in surgical performance. However, to obtain reliable 
individual differences in rates of complication from surgeries it is necessary 
to evaluate the outcomes of hundreds or thousands of hours of surgery for a 
given surgeon. Even if all the chess playing and surgery was recorded on 
video tape it would be nearly impossible to analyze all those hours of tape to 
find particular aspects of performance that accounted for individual 
differences in outcomes. Furthermore, essentially no chess game is exactly 
the same as any other game, and two patients never have exactly the same 
medical problem. Although it is possible to statistically control for many of 
these factors in analyzing aggregate outcomes, it would be very difficult to 
search for critical individual differences in processes accounting for superior 
outcomes from analysis of real-world events. 

A major breakthrough for the study of expert performance with protocol 
analysis was made by de Groot (1978) in the study of chess. He identified 
challenging chess positions from actual games of chess masters. He was then 
able to present the positions one at a time to individual chess players of 
different skill levels for selection of the “next best move.” Subsequent 


research has shown that this method of presenting a series of representative 


situations and requiring generation of appropriate actions provides the best 
available measure of chess skill for predicting performance in chess 
tournaments (Ericsson, Patel, & Kintsch, 2000; van der Maas & 
Wagenmakers, 2005). The issues of calculating correlations to show that 
performance on a particular type of standardized task captures real-world 
performance are discussed in more detail in a later chapter in this volume 
(Ericsson, Chapter 38, this volume). Furthermore, this methodology makes it 
possible for experts to think aloud while selecting their moves which allows 
unique insights into their thought processes. If the chess master would think 
aloud during a tournament match the opposing player would know what they 
were planning to do. Presenting chess problems with think-aloud protocols 
can be completed in 15-20 minutes and thus can capture chess skill much 
more effectively and offer much better potential for identifying mechanisms 
mediating chess skill than an analysis of hundreds of hours of chess playing. 
Before describing how this methodology can be applied to a wide range of 
other domains of expertise, I will describe the pioneering research conducted 


in chess. 


The pioneering studies of expertise in playing chess 


De Groot (1978) had identified chess positions that he had analyzed for a 
very long time to establish the best move. The best move can today be 
objectively determined by chess-playing computers, which are clearly 
superior even to the best human players. When players are thinking aloud 
while selecting the best move for a chess position they first familiarize 
themselves with the position and verbalize salient and distinctive aspects of 
the position along with potential lines of attack or defense. The players then 
explore the consequences of particularly promising potential moves by 
planning alternative move exchanges and then evaluating the resulting 
positions. 

De Groot’s (1978) analysis of the protocols identified two types of 
differences in cognitive processes, which could explain the superior selection 
of moves by the world-class players compared to club players. The world- 
class players verbalized many good first moves even during their initial 
familiarization with the chess position. For example, they would notice 
weaknesses in the opponent’s defense that suggested various lines of attack 
and then examine and systematically compare the consequences of various 
sequences of moves. During this second phase of analysis, these world-class 
players would often discover new moves which were the best moves and 
superior to all the previously generated ones. In contrast, the less skilled 
players did not even verbally report thinking about the move that was 
determined to be the best available move, suggesting that they did not, in 


fact, consider that move. Thus their initial inferior representation of the 


position did not allow them to notice the value of lines of play starting with 


that move. 


Mechanisms mediating higher levels of chess expertise 


De Groot’s informal analysis of the protocols revealed the mechanisms and 
processes of the world-class players, which could account for their 
superiority in finding and selecting moves. 

The first difference involved the best players’ ability to rapidly perceive 
the relevant structure of the presented chess position, thus allowing them to 
identify weaknesses and associated lines of attack that the less accomplished 
players never reported noticing in their verbal protocols. This superiority of 
skilled chess players in rapidly selecting chess moves has been demonstrated 
in studies of the move quality when chess players play speeded chess, where 
the available time for making each move is very short (roughly around 5 to 


10 percent of the time available in normal chess) (Burns, 2004; Calderwood, 


Klein, & Crandall, 1988). Converging evidence was reported by Moxley, 


Ericsson, Charness, and Krampe (2012) from a study of participants, who 
were thinking aloud while trying to select the best move for a series of 
positions without any time pressure. Moxley et al. (2012) found that the first 
mentioned move by more skilled players in the beginning of the protocol was 
rated better by a chess computer than that of less skilled players. 

Highly skilled players also show a superior ability to generate and 
evaluate the consequences of potential moves by planning, which goes 
beyond the initial recognition of promising moves. De Groot (1978) found 
that during the planning and evaluation process, the world-class players often 
discovered better moves than those they had perceived initially during the 
familiarization phase. In a more recent study, Charness (1981) collected 


think-aloud protocols from experts and less skilled chess players on the 


selection of a move for a chess position not studied by de Groot. Examples of 
an analysis of the protocols from a club-level player and an expert-level chess 
player are given in Figure 12.2. Consistent with these examples Charness 
(1981) found that the depth of planning increased with greater chess skill. In 
addition, there is evidence that increased time available for planning 
increases the quality of the moves selected, where the quality of selected 
moves during regular chess is superior to those selected during speed chess 
(Chabris & Hearst, 2003). Perhaps the most compelling evidence for the 
benefits of planning comes from the think-aloud analyses by Moxley et al. 
(2012). They measured the quality of the first move mentioned and compared 
it to the quality of the move that was finally selected as the players’ choice. 
For the skilled players the benefit of planning was observed for the difficult 
chess positions, because they were able to pick good moves initially for the 
easy problems. The less skilled players gained most from planning for the 


easy problems. 


White to Move (P-c5) 


Expert 


Considered Advance white 
next move pawn on cf to c5 
le 


and 


Black pawn on dé will Move black knight on 

take that white pawn d7 to e5 

and 
Move white love white ve 
knight on c3 i c i 
to bi ] 0 


White pawn on b4 
will take that black 
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Figure 12.2 A chess position presented to chess players with the 
instruction to select the best next move by white (top panel). The think- 
aloud protocols of a good club player (chess rating = 1657) and a chess 
expert (chess rating = 2004) collected by Charmess (1981) are shown in the 
bottom panel to illustrate differences in evaluation and planning for one 
specific move, P-c5 (white pawn is moved from C4 to c5), which is the best 
move for this position. Reported considerations for other potential moves 
have been omitted. The chess expert considers more alternative move 
sequences and some of them to a greater depth than the club player does 
(from Ericsson & Charness, 1994, Figure 1, copyright American 
Psychological Association). 


In support of a superior ability of skilled chess players to image chess 


positions mentally, investigators have examined their ability to play 


blindfold, without a visible board showing the current position. Chess 
masters, but not less skilled players, have demonstrated the ability to play 
chess at a relatively high level under blindfold conditions (Chabris & Hearst, 
2003; Karpov, 1995; Koltanowski, 1985). In more controlled experiments the 
investigator announced moves from a chess game one move at a time Starting 
from the beginning of a game. After a series of announced moves the 
investigator would stop reading and the players would then have to recall all 
the locations of pieces at that point in the game (Ericsson & Staszewski, 
1989). Experiments show that chess masters are able to mentally generate the 
chess positions associated with multiple chess games without any external 
memory support when the experimenter reads sequences of moves from 
multiple chess games (Saariluoma, 1991, 1995). 

In sum, the analyses of the protocols for the move selection task 
uncovered two different mechanisms that mediated the expert chess players’ 
superior ability. Upon the initial exposure to the chess position they were able 
to recognize patterns and retrieve appropriate moves for easy problems. For 
more difficult move selection problems the expert players need time to 
engage in planning of consequences of potential moves to find better moves. 
As their chess skill increases, they become increasingly able to encode and 
manipulate internal representations of chess positions to plan the 
consequences of chess moves, discover potential threats, and even develop 


new lines of attack (Ericsson & Kintsch, 1995; Saariluoma, 1992). For a 


discussion of the relation between the superior memory for presented chess 
positions and the memory demands integral to selecting chess moves see 
Gobet and Charness (Chapter 31, this volume) and Ericsson (Chapter 36, this 


volume). 


Applications to Expert Performance Using 
Protocol Analysis in Other Domains 


There are several challenges for successfully applying this approach to other 
domains. Ideally one should be able to identify representative situations from 
the domain that require immediate action. Behavior in these situations needs 


to capture the essence of superior performance in the associated domain. 


Other Games 


There are other domains that resemble chess, where a similar methodology 
has been adapted. In many types of games, such as poker and Scrabble, 
participants confront challenging situations where computers allow the 
identification of optimal and superior actions. In Scrabble there is a rating 
system for skill similar to chess and Tuffiash, Roring, and Ericsson (2007) 
captured superior skill indicators by having participants select the best action 
for a set of Scrabble situations while they were thinking aloud. An analysis of 
the protocols showed that more skilled participants generated more word 
alternatives with higher point values. Poker players differing in skill were 
thinking aloud while finishing 60 pre-selected hands in Texas Hold’em while 
playing against a computer. St. Germain and Tenenbaum (2011) found that 
the expert players made significantly better decisions at all stages of playing 
out the poker hand than the novices. The expert players verbalized more 
thoughts than novices, in particular more advanced considerations of the 


opponents’ anticipated behaviors. 


Medicine 


The fundamental goal in medicine is to treat patients to increase their health 
and well-being in the short- and long-term. It is difficult to provide 
immediate feedback, when many of the valid outcomes for patients require 
long-term follow-ups. One possibility would be to collect videos and detailed 
records of particular patients at the time of diagnosis and treatments and then 
wait until a final valid diagnosis is attained along with a long-term outcome 


(Ericsson, 2015). In particular, there are diagnostic tests, such as X-rays, 


electrocardiograms (ECGs), and analyses of cells for microscopic pathology, 
which correspond to documents that can be saved for re-presentation at times 
when the correct diagnosis is available. 

Diagnosing breast cancer from reviewing mammograms can be 
evaluated by the emergence of cancer within a couple of years. It is therefore 
possible to measure individual differences in diagnostic accuracy 
retrospectively and these differences have been found to be large (Miglioretti 
et al., 2009). The probability of cancer is, however, very low (less than 1 in 
1,000 examined mammograms) so when researchers developed tests of 
diagnostic ability during medical training they would present X-rays from 
previous patients with known outcomes. Furthermore they would typically 
select a comparable number of pathological and normal cases. When Nodine 
et al. (1999) tested radiology residents and fellows and their supervisors the 
accuracy of the diagnoses of an individual increased as a logarithmic function 
of the number of mammograms that that individual had encountered during 
their professional experience. Experienced radiologists reached a stable 


accuracy level (far from perfect) at around 10,000 mammograms — the 


number of mammograms completed by the supervising radiologists. Highly 
experienced and less experienced radiologists have been asked to think aloud 


while making their diagnoses (Azevedo, Faremo, & Lajoie, 2007; Lesgold et 


al., 1988) and their thoughts have been compared to those of less experienced 
trainees. Based on an analysis of the think-aloud protocols there is clear 
evidence that the cognitive processes shown by those with more experience 
are associated with superior representations. Higher skilled individuals were 
more able to self-regulate and apply successful search and reasoning 
strategies from their more refined mental representations. 

In medicine there are many other types of medical images that require 
perceptual diagnosis. For example, Crowley, Naus, Stewart, and Friedman 
(2003) presented microscopic slides of stained tissue with known presence or 
absence of breast cancer to medical students, pathology residents, and 
experts. Experts were found to be more able to integrate the information into 
an accurate diagnosis than residents and students. Experts were more able to 
correctly identify anatomical locations than students and more able to classify 
lesions than residents. Jaarsma, Jarodzka, Nap, Merriénboer, and Boshuizen 
(2014) presented digitally scanned microscopic images of colon tissue to 
medical students, residents, and experts, but were only able to find significant 
differences in performance between the medical students and the two more 
skilled groups. The two more skilled groups differed from the novices in 
ways consistent with Crowley et al. (2003). In a similar study of ECGs 
Gilhooly et al. (1997) elicited interpretations from medical students, 
residents, and experts and found increased accuracy with higher levels of 
expertise. The analysis of the think-aloud protocols showed that the primary 
advantage of experts concerned their superior ability to reason about 


observable features of the ECG and infer an electrophysiological model to 


distinguish between alternative diagnostic hypotheses. Finally, Balslev et al. 
(2011) collected think-aloud protocols from 43 clinicians with varying 
amounts of professional experience, while they diagnosed neurological 
seizures from brief video tapes. Longer professional experience was 
associated with superior diagnostic accuracy and a higher percentage of 
clauses in the protocols referring to generating and evaluating diagnostic 
hypotheses. 

Interpretation of medical images is just one aspect of diagnosing and 
then treating patients, where the superior performance, discussed above, is 
demonstrated by medical doctors specializing in interpretation of particular 
types of images. The most common diagnostic situation involves a patient 
presenting a problem to a doctor, who is asking questions and assessing 
individual differences in weight, temperature, mental status, and skin color, 
along with observations of heart and lung sounds. The interaction between a 
doctor and a patient is difficult to standardize and most of the research on 
medical diagnosis has relied primarily on verbal descriptions of hypothetical 
patients based on simplified summaries of actual patients. Most of this 
research has not tried to capture real-world performance but the focus has 
been on eliciting differences between students, advanced students, and 
experts. Boshuizen and Schmidt (1992) presented participants with a single 
description of a patient and found that the process of generating a diagnosis 
becomes more efficient as medical students complete more of their medical 
training. The increase in efficiency is mediated by higher levels of 
representation that is acquired to support clinical reasoning (Schmidt & 
Boshuizen, 1993). Differences in diagnostic thinking as a function of 
expertise differs not only for relatively “common and _ straightforward 


problems” (Norman, Trott, Brooks, & Smith, 1994, p. 119) but also for more 


complex and challenging problems. For the common problems there is 
frequently no difference in accuracy for residents and experts with only a 
difference in the process, where experts are more able to rely on recognition 
and shortcuts. For example, McLaughlin, Novac, Rikers, and Schmidt (2010) 
found that nephrologists were superior to students in diagnosing patients with 
electrolyte problems while thinking aloud. For students there was a striking 
increase in solution accuracy with a greater number of verbalized biomedical 
concepts, whereas no such relation was observed for the experts. A different 
pattern would be expected for more complex problems, because recognition 
is not a viable option in solving unfamiliar complex problems (Ericsson, 
2004). In an influential study Norman et al. (1994) presented eight complex 
nephrology problems and asked participants to think aloud. They found large 
differences in accuracy between nephrologists and residents. Norman et al. 
(1994) found that experts relied on the encoding of meaningful relations 
between laboratory data and “more extensive use of causal explanations” (p. 
114). More recently, McRobert et al. (2013) had experienced emergency 
physicians and residents think aloud while diagnosing patients in two 
scenarios in a high-fidelity simulation. The skilled group verbalized more 
evaluation, predictions, and deep plans consistent with a more refined 
representation for generating and evaluating options. 

A common alternative to instructing participants to think aloud while 
they generate their diagnoses, is to instruct them to first generate a diagnosis 
and then generate an explanation and rationale for their diagnosis. These 
processes are clearly different, but should both reflect the underlying mental 
representations and knowledge. Several studies have been able to show large 
differences in diagnostic performance as a function of expertise. The 


specialists have been found to generate explanations that are more complete 


and provide better organized evidence in support of their diagnoses (Patel & 
Groen, 1991). The experts’ superior reasoning ability appears to reflect 
higher-level representations that they have acquired to support reasoning 
about clinical alternative diagnoses (Ericsson & Kintsch, 1995; Ericsson et 


al., 2000; Patel, Arocha, & Kaufmann, 1994), 


Sports 


The expert performance approach has been applied to a wide range of 
domains, where skilled and less skilled performers solve representative 
problems while thinking aloud. When the review is restricted to studies in 
domains that show reproducibly superior performance of experts, the think- 
aloud protocols reveal patterns of reports that are consistent with those 
observed in previously reviewed domains. For example, when expert snooker 
players are instructed to make a shot for an arranged configuration of pool 
balls they verbalize deeper plans and more far-reaching exploration of 
consequences of their shots than less skilled players (Abernethy, Neal, & 
Koning, 1994). While engaging in the self-paced task of putting on a practice 
green Calmeiro and Tenenbaum (2011) instructed three experienced and 
three beginner golfers to think aloud. In a more recent laboratory study 
Whitehead, Taylor, and Polman (2015) had 30 skilled and 30 novice golfers 
think aloud and found significant differences in putting performance between 
the groups along with an increased frequency of the skilled golfers 
verbalizing gathered information about the putt and plans for executing the 
putt — a similar pattern to the one observed by Calmeiro and Tenenbaum 
(2011) and Arsal, Eccles, and Ericsson (2016). Recently, this same relation 
between increased planning and higher skill has been found when high 
skilled, medium skilled, and low skilled golfers think aloud while putting a 
series of holes on a golf course (Whitehead, Taylor, & Polman, 2016). 

The majority of team sports require athletes to make fast decisions and 
execute rapid responses to dynamically changing situations. There are studies 


that collected think-aloud protocols during actual games, such as during the 


changing of sides in tennis (McPherson & Kernodle, 2007). However, most 
applications studying expertise have involved presentation of standardized 
game situations using videos or scripted situations with actual players and 
requested a rapid response followed by immediate retrospective reports. A 
few studies have asked the athletes to generate solutions concurrently to 
defensive game situations in baseball (French et al., 1996). In a meta-analysis 
Mann, Williams, Ward, and Janelle (2007) found that the performance 
advantage for experts increases when the test situation replicates the actual 
situation during competition. Consistent with this finding Afonso, Garganta, 
McRobert, Williams, and Mesquita (2014) found that the verbal reports from 
the scripted actual situations contained greater sophistication, namely more 
concern for opponents, than in the video condition. Similarly Roca, Williams, 
and Ford (2013) found more extensive verbal reports when soccer players 
were allowed to move freely in front of a life-size video screen versus a 
condition where they had to remain seated in front of the same screen. In an 
interesting study Roca, Ford, McRobert, and Williams (2011) produced short 
video sequences from soccer games from a particular player’s perspective 
and instructed the more and less skilled soccer players to predict what would 
happen after the end of the clip and what would have been the best action. 
The skilled players were more accurate than the less skilled and their 
retrospective reports consisted of a higher proportion of evaluation, 
prediction, and planning statements. It will be important to show that these 
general findings about the thinking of expert performers generalize to 
performance under the stress found in actual competitions as Whitehead et al. 
(2016) have done. 


Other applications 


In this handbook there are many chapters that describe and discuss 
applications of verbal reporting methodologies to study thinking in several 
different domains of expertise with clearly defined objective performance, 
such as chess (Gobet & Charness, Chapter 31), mathematics (Butterfield, 
Chapter 32), music (Lehmann, Gruber, & Kopiez, Chapter 28), and medicine 
and surgery (Norman et al., Chapter 19). There are also reviews of other 
domains with lessened criteria for superior performance, such as nursing 


(Lundgrén-Laine & Salanteré, 2010) and entrepreneurial expertise (Dew, 


Ramesh, Read, & Sarasvathy, Chapter 22, this volume). Some of the most 
important advances from the application of verbal report methodology have 
been in domains that involve creativity and less well-defined criteria for 
expert performance. A very nice example is given by Fayena-Tawil, Kozbelt, 
and Sitaras (2011) comparing think-aloud verbalizations from artists and non- 
artists while engaged in an open-ended drawing task (see also Kozbelt & 
Ostrofsky, Chapter 30, this volume). Think-aloud verbalizations have 
similarly been very important in understanding characteristics of thought 
processes of professional designers (Cross, Chapter 21, this volume) as well 
as music composers (Collins & Dunn, 2011) and advanced music students 


while they practice (Nielsen, 1999). 


Conclusion 


Protocol analysis of thoughts verbalized during the experts’ superior 
performance on representative tasks offers a rigorous alternative to the 
traditional subjective methods of directed questioning and introspection. This 
theoretical methodological framework for collecting verbal reports related to 
observable performance differences has led to an accumulation of evidence 
on expert performance. This body of evidence has convinced many scientists 
to accept this type of data related to cognitive constructs, such as memory 
and reasoning. Even behaviorists, who have a long history of criticizing 
introspection and mental constructs, have now embraced the methodology to 
account for rule-governed behavior (Austin & Delaney, 1998). More 
generally, the methodology of protocol analysis has emerged as the primary 
tool that allows researchers to identify thoughts and information that pass 
through expert and other types of performers’ attention while they generate 
their behavior. In support of this claim, “think-aloud” protocols have 
emerged as a practical tool to describe and diagnose thinking outside of 
traditional laboratory studies in cognitive psychology and cognitive science. 
For example, researchers and designers of surveys and questionnaires (Willis, 
2005, 2015) on cognitive processes mediating answers to existing 
questionnaires (Darker & French, 2009) rely on this methodology, which is 
also adaptable for usability testing of software (Cooke, 2010; MacDonald, 
Edwards, & Zhao, 2012). The methodology of “think aloud” and immediate 
retrospective reports is one of many different methods used in the new 


approaches combining quantitative and qualitative methods referred to as 


mixed methods (Carayon et al., 2015). There are many new methods for 
eliciting verbalizations of thoughts that differ from the traditional think-aloud 
method (Fox et al., 20i1a). In fact, when participants were instructed to 
explain their behavior their overall performance was improved, which 
suggests that explaining one’s behavior can be used as a method to make 
education more effective (Fox et al., 20i1a). Other investigators have asked 
participants to give descriptions rather than simply “think aloud.” For 
example, Den Hartigh et al. (2014) had soccer players of different levels of 
expertise “describe (aloud) the actions taking place on the field” (p. 758) 
when watching a video of a soccer game and found that skilled players 
reported more relevant events and actions away from the ball than the less 
skilled players. Similarly, Chaffin, Imreh, Lemieux, and Chen (2003) studied 
the concurrent commentary and actual practice behavior of a professional 
pianist mastering a music piece for public performance. There is a large range 
of different types of instructions to verbalize that will provide important tools 
for assessment and, in particular, for allowing teachers the ability to monitor 
and provide feedback on thinking during the students’ learning and training. 
The complexity and diversity of the mechanisms mediating skilled and 
expert performance are frequently intimidating. To meet these challenges it is 
essential to develop methods that allow investigators to collect rich data that 
can provide insights into the structure of an individual expert’s superior 
performance, when the performance is repeatedly reproduced under 
controlled and experimental conditions. Process tracing, in particular protocol 
analysis, will be essential for uncovering detailed information about most of 
the important mechanisms that are responsible for the superiority of the 
experts’ achievement and the experts’ ability to keep improving these 


mechanism during designed practice. Only then will it be possible to discover 


their structure and study their development and refinement with training and 


deliberate practice. 
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Introduction 


“Psychometrics” refers to the scientific discipline that combines 
psychological inquiry with quantitative measurement. While psychometric 
theory and practice pertain to all aspects of measurement, in the current 
context, psychometric approaches to expertise pertain to the measurement 
and prediction of individual differences and group differences (e.g. by 
gender, age) and in particular, high levels of proficiency including expertise 
and expert performance. The scientific study of expertise involves several 
important psychometric considerations, such as reliability and validity of 
measurements, both at the level of predictors (e.g. in terms of developing 
aptitude measures that can predict which individuals will develop expert 
levels of performance), and at the level of criteria (the performance measures 
themselves). We will discuss these basic aspects of psychometric theory first, 
and then we will provide an illustration of psychometric studies that focus on 
the prediction of expert performance in the context of tasks that involve the 
development and expression of perceptual-motor skills, and tasks that involve 
predominantly cognitive/intellectual expertise. Finally, we will discuss 
challenges for future investigations. 

A clear definition of terms is essential for understanding the concepts in 
this chapter. “Traits” refer to relatively broad and stable dispositions, which 
can be physical (e.g. visual acuity, strength) or psychological (e.g. 
personality, interests, intelligence). In contrast to stable traits, states represent 
temporary characteristics (e.g. sleepy, alert, angry). “Inter-individual 


differences” refer to differences between individuals, such as the difference 


between the heights of students in a classroom or the speed of different 
runners in a race. In psychometric theory, understanding inter-individual 
differences is essential for understanding how a test might differentiate 
between people who have high levels of a trait or state and people who have 
lower levels. And, because one major purpose of psychological assessments 
is to discriminate between people on a trait, inter-individual differences are 
central to psychological measurement and psychometrics. 

Intra-individual differences refer to differences within individuals, such 
as the difference in scores on a typing-speed assessment administered at the 
beginning of a typing class and that same assessment administered to the 
Same person at the end of a year of practice. Studies of expert performance 
and the development of expertise during skill training can focus on either 
inter-individual differences (e.g. the rank-ordering of expert chess players, or 
a group of trainees acquiring expertise in an area), intra-individual 
differences (e.g. measuring the transition of performance from novice to 
expert performance), or a combination of the two (e.g. inter-individual 
differences in intra-individual change), which is an assessment of the 
differences in people that are related to growth or decline in skills over time. 
For instance, an assessment of inter-individual differences in intra-individual 
change would be used to examine which trainees (i.e. based on their traits) 


learn the most or the least over the course of training. 


General Aspects of Psychometric Approach: 
Predictors 


Two fundamental aspects of measurement transcend psychology and other 
scientific inquiries: reliability and validity. We discuss each of these in turn 


below. 


Reliability 

At a general level, the definition of psychometric reliability is not different 
from a common-sense meaning of the term. If one co-worker shows up for 
work at nearly the same time every day, you might say that her attendance is 
reliable. If a different co-worker is often late or even sometimes early, but 
you can rarely predict when she will actually walk through the door of the 
office, you might consider her to be unreliable in attendance. The 
psychometric concept of reliability concerns a similar accounting of 
consistency and precision, except in this case we ordinarily refer to the 
reliability of measures or tests, rather than individuals. A test or other form of 
assessment is considered to yield reliable results when a group of people can 
be consistently rank-ordered over multiple measuring occasions. 

There are an array of approaches for estimating the reliability of a 
measure, some more suitable to particular occasions than others. The test- 
retest method involves administering the same test on two different occasions 
and then comparing the rank-ordering of people on those two administrations 
(i.e. correlating the outcomes of the two measurement occasions). For 
example, a test of running speed (i.e. a relatively stable trait) might involve 
measuring how fast a group of runners can run 10 km. To assess the 
reliability of this test, a researcher would simply administer the test on two 
occasions and correlate the results. However, in this particular case, the 
timing of the tests would be critical: If the second test was given immediately 
following the first, performance might be very different across the two tests 
for reasons more related to states (i.e. differential fatigue) than running speed, 


the trait under investigation. Thus, the results of such an administration might 


erroneously suggest that the test is not very reliable. Furthermore, individual 
states such as mood, amount of sleep the previous night, and so on may affect 
performance on both testing occasions, which would confound the measure 
of running speed with an individual’s state. In this example, then, a reliability 
estimate that involves separate measurement occasions would relegate 
momentary fluctuations such as fatigue and recovery to measurement error. 
The reliability estimate from such a procedure would reflect the measurement 
consistency for the enduring running speed trait. 

Although obtaining test-retest reliability estimates is practical and 
reasonable for some performance measures, there are two main problems for 
using test-retest procedures for estimating reliability of psychological 
(cognitive, personality, affective) traits. The first problem is memory — 
humans may remember how they responded to a survey or test across 
occasions, unless the tests are separated by a very long time (and sometimes, 
even this is insufficient). The second problem pertains mostly to performance 
measures, such as aptitude, ability, and skill assessments. This problem is 
learning; that is, examinees often learn either explicitly or implicitly during 
the test. Tests ranging in difficulty from simple arithmetic problems to 
complex simulations typically show significant and sometimes substantial 
improvements in performance from one occasion to the next, either because 
examinees have learned the correct responses, or because they have become 
more skilled at performing the basic operations required by the test. Under 
these conditions, a more appropriate method of assessing the reliability of the 
test is to use what is known as an “alternate form.” In this approach, instead 
of administering the same test on two occasions, an alternate form of the test 
is given on the second administration. The alternate form is typically 


designed to be similar to the first in terms of content and psychometric 


properties related to difficulty and validity, but it differs in terms of the actual 
items presented to the examinee. When fatigue is not an important 
consideration, alternate forms of a test can be administered one right after the 
other. Otherwise, alternate forms can be administered after a delay, similar to 
the test-retest procedure described above. 

A final type of reliability relevant to the study of expertise is inter- 
observer reliability. This is an index of agreement between different judges, 
when performance cannot be objectively evaluated (e.g. gymnastics, diving, 
art, music). When judges have high agreement in rank-ordering individuals, 
there is high inter-observer reliability; but when there is litthe agreement, 
reliability of the judgments is low. 

Reliability of a measure is the first hurdle that must be passed for it to be 
scientifically or practically useful. Without reliability, a test has little or no 
utility. But, just having a consistent rank-ordering of individuals on a test 
says nothing about whether or not the test actually measures what it sets out 
to measure. For that assessment, we have the concept of validity. And even 
though reliable measurement is essential for establishing the validity of a 
measure, reliable measures may not be valid for any particular application or 
theory. Thus, we will follow up our discussion of reliability with a review of 
the critical considerations of validity. The final part of this section will 
consider issues of reliability and validity in terms of predicting individual 
differences in expert-level performance, especially in light of base rate 


concerns. 


Validity 

Validity is a property of an instrument that refers to whether it measures what 
it sets out to measure. Thus, a test of baseball skill is valid to the degree that 
the inferences made about a person’s underlying baseball ability using this 
measure are valid. Validity is generally discussed in terms of construct, 
content, and criterion considerations. Construct validity — or whether a test 
measures what it purports to measure — is the central validity question. It is 
typically assessed by evaluating the correlation between the measure and 
other assessments of the same or similar constructs. Generally, in order for a 
measure to have high construct validity, it must correlate substantially with 
other measures of the same or similar constructs (this is called “convergent 
validity”), and the measures should not correlate substantially with measures 
of different constructs (this is called “discriminant validity”). For example, a 
measure of general baseball skill might be expected to have high correlations 
with a measure of baseball strategy knowledge (convergent validity), but low 
correlations with a measure of football strategy knowledge (discriminant 
validity). 

Content validity refers to the appropriateness of the domain sampled in 
the assessment, which is generally dictated by the purpose of the test. For an 
assessment of general baseball skill, the appropriate content of the skill 
would include batting, running, and fielding, and these skills would be 
assessed to the same degree that they are important for baseball performance. 
By contrast, a test of baseball pitching skill would be narrower in the content 


sampled, focusing only on pitching speed and the types of pitches executed. 


Generally, content validity is established through judgments of subject matter 
experts, and not in a quantitative fashion. 

Especially important in terms of the application of psychometric 
measures is criterion-related validity. The key to criterion-related validity is 
prediction; it refers to the degree to which the measure can predict individual 
differences in an external criterion. The typical application-oriented goal is to 
use a test or other assessment (e.g. interview) as an aid in selection, such as 
for education or training opportunities, for a job, or to be a team member. For 
example, criterion validity for an intelligence test is frequently demonstrated 
by examining the degree to which scores on the intelligence test correlate 
with a criterion of academic performance, such as grade point average or 
academic promotion from one grade to the next. Ideally, criterion-related 
validity is assessed by administering the test to a group of individuals who 
are all selected for the educational or occupational opportunity. Criterion 
performance is then measured at a later time, such as after training, or after a 
period of job performance. This kind of assessment is called “predictive 
validity” and it provides the most precise estimate of the relationship between 
the measure and the criterion, although this approach is negatively affected 
when individuals leave the program prior to the time that criterion 
measurement is obtained. 

When it is not possible to use such a procedure (such as when there is 
already some selection procedure in place), an investigator can perform a 
concurrent-validation assessment. In this procedure, the measure is 
administered to incumbents (e.g. current employees, current students, current 
team members), and their criterion performance is assessed in the same 
testing administration (i.e. simultaneously). Given that one can usually 


assume that incumbents are more restricted in range on the key traits for 


performance than are applicants (through existing selection procedures, self- 
selection, or attrition associated with training/performance failures), 
establishing the validity of a new measure is more difficult using concurrent- 
validity procedures than it is for predictive-validity procedures. Nonetheless, 
the ease of executing this approach — particularly because all measures are 
gathered at one time — makes it a popular choice for establishing validity. 
Procedures exist for estimating the predictive validity of a test when assessed 
in a concurrent-validity study, especially when there are data concerning the 
differences between incumbents and applicants (e.g. see Thorndike, 1949). 
For example, aptitude tests such as the SAT only show relatively modest 
concurrent-validity correlations with college grade point average at selective 
colleges and universities. However, an institution can estimate the predictive 
validity of the SAT, given knowledge of the test score distributions of both 


applicants and incumbents to obtain a more realistic validity coefficient. 


Special Considerations of Measurement in 
the Prediction of Expert Performance 


By its very nature, the study of expertise is associated with several specific 
measurement problems. We consider four of the most important problems: 
measurement of change over time, restriction of range, base rates, and 


interdependence issues. 


Measurement of Change over Time 


From early in the 1900s, psychologists interested in individual differences in 


learning and skill acquisition (e.g. Thorndike, 1908; see Ackerman, 1987; 


Adams, 1987 for reviews) have attempted to evaluate which traits are 


associated with learning the most during task practice or from training 
interventions. Learning can generally be considered to be a relatively 
permanent change in behavior over time and thus assessing learning requires 
assessing change, which can be complicated. Measurement artifacts related to 
regression to the mean and the underlying nature of individual differences in 
learning make this task difficult. Regression to the mean is a Statistical 
phenomenon, not a set of causal effects. When measurements (in this case, 
initial performance on a task) are not perfectly reliable, those individuals with 
extreme scores on the first occasion are likely to obtain scores closer to the 
respective mean for the second occasion (after task practice). This means 
that, ceteris paribus (i.e. if everything else is equal), individuals with below 
average scores at initial performance measurement will have relatively higher 
scores at the second occasion, and individuals with above average scores on 
the first occasion will have relatively lower scores on the second occasion. 
Again, this is a statistical phenomenon, but it results in potentially critical 
artifacts that can be misinterpreted as meaningful change over time. 

The deeper problem occurs when a researcher attempts to evaluate the 
relationship between initial task performance and the amount of learning (or 
gain in performance) after practice or training. Given the nature of the 
regression to the mean phenomenon, the expected correlation between initial 


performance and later performance will be negative (simply as a function of 


the regression to the mean). An unsuspecting researcher might be tempted to 
conclude that a training program has the effect of “leveling” individual 
differences in performance, in that the poor performers get relatively better 
and the good performers get relatively worse (McNemar, 1940). Ultimately, 
it is a bad idea to attempt to measure individual differences in learning by 
correlating initial performance with performance after practice or training 
(e.g. see Cronbach & Furby, 1970). A better approach would be to include 
multiple measurement occasions (i.e. at least three) and model change over 
time using random coefficient modeling (hierarchical linear modeling; Byrk 
& Raudenbush, 1992; Ployhart & Vandenberg, 2010), and/or latent growth 
modeling (Ployhart & Hakel, 1998). The advantage of these approaches over 


more traditional methods for measuring change is that they partition the 
variance into within person (intra-individual) and between person (inter- 
individual) components, permitting the assessment of systematic performance 
trends over time within person (e.g. linear, quadratic, and cubic) and 
between-person traits that can predict these trends (inter-individual 
differences in intra-individual change). 

A second issue related to measuring individual differences in learning is 
the nature of inter-individual variability during learning or skill acquisition. 
Because the magnitude of inter-individual variability is associated with 
changes in the reliability and validity of predictor measures, it is important to 
take account of factors that might lead to changes in inter-individual 
variability. For skills that can be acquired by all or nearly all learners (i.e. 
simple and consistent tasks), inter-individual variability tends to decline with 


task practice or training (e.g. see Ackerman, 1987); frequently, the changes in 


variability can be substantial. For example, tasks with substantial motor or 


perceptual-motor components, such as typing or golf, have extremely large 


inter-individual differences in initial performance, but performance 
variability decreases with training. One reason for this is that there are 
physical limitations on performance at high levels of expertise: the most 
expert typist can only type as rapidly as one keystroke every 100 ms, and the 
most expert golfer is likely to perform a handful of strokes under par. By 
contrast, there are few limits at the other end of the performance continuum — 
there are many more ways that an individual can perform a task poorly than 
there are ways that a task can be performed at an expert level. Thus, when it 
comes to comparing the learning rates of a group of individuals, it is the 
poorest performing learners that have the most to gain, and the highest initial 
performers who have the least to gain from task practice or training. 

It is important to emphasize that these substantial changes in inter- 
individual variability are typically found only for tasks that are within the 
capabilities of nearly all learners. When tasks are complex or inconsistent in 
information processing demands, inter-individual variability may not change 
over task practice (Ackerman, 1987, 1992; Ackerman & Woltz, 1994), or 
there may be a Matthew effect, which is when those who start with the most 
skill tend to learn the most (e.g. the “rich getting richer,” see Stanovich, 
1986). Such effects have been found in reading skills and in other cognitive 
or intellectual tasks (Lohman, 1999). For example, expertise in mathematics 
is likely to show a Matthew effect, as many learners effectively drop out of 
the learning process at different stages of developing expertise (e.g. at the 
level of acquiring skill at algebra, at calculus, or beyond). Across normal 
development, the differences between experts and non-experts in 
mathematics will become more pronounced, which will be manifest as larger 
inter-individual variability in performance after practice or education. 


A few laboratory-based examples may help illustrate the nature of the 


development of expert performance in the context of changing mean 
performance and changes in inter-individual variability (as expressed in the 
between-individual standard deviation of performance). Figure 13.1(a)—(c) 
shows three different tasks, which are reasonably well defined, but differ in 
the nature of the task demands and the effects of practice on task 
performance. The first graph is from a skill acquisition experiment with a 
simplified Air Traffic Controller (ATC) task (the Kanfer-Ackerman Air 
Traffic Controller task; Ackerman & Cianciolo, 2000; for a more extensive 


description of the task, see Kanfer & Ackerman, 1989). The task is difficult 


for participants when they first perform it. Cognitive/intellectual abilities 
(discussed below) are substantially related to individual differences in initial 
task performance. However, the task has only seven rules, and all task 
operations can be accomplished with just four different keys on the computer 
keyboard. As a result, within five or six hours of practice, nearly all of the 
learners become expert performers. Figure 13.1(a) shows how mean 
performance increases quickly in the early sessions of practice, but becomes 
asymptotic as most learners develop high levels of skills. Between-individual 
standard deviations start off high (when there are large differences between 
those learners who easily grasp the task demands early in practice, and those 
learners who must struggle to keep up), then decline as the slower learners 
ultimately acquire the skills necessary to perform the task at an expert level. 
At the end of six hours of practice, the magnitude of between-individual 
standard deviations has changed from 9.08 to 6.55, a reduction of about 28 


percent. 
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Figure 13.1(a)-(c) Performance means and between-individual standard 
deviations over task practice. Panel (a) Kanfer-Ackerman Air Traffic 
Controller task (data from Ackerman & Cianciolo, 2000); Panel (b) 
Terminal Radar Approach Control task (data from Ackerman & Kanfer, 
1993); Panel (c) Noun—Pair Lookup task (data from Ackerman & Woltz, 
1994). 


In Figure 13.1(b), a more high-fidelity ATC task was used. In contrast to 
the previous task, this one (called TRACON, for Terminal Radar Approach 
Control; see Ackerman & Kanfer, 1993) involves sustained and focused 
attention, continuous sampling of the visual radar screen, short-term memory 
and problem-solving abilities, and spatial visualization. There are many more 
commands to learn, and each 30 minute task trial involves novel 


configurations of airplanes that the learner must handle in real time. Few 


participants perform very well on the first few trials, and in general, it takes 
much longer to acquire skills on TRACON than it does for the simpler 
Kanfer-Ackerman ATC task. For TRACON, many learners do not reach 
expert levels of performance, even after extensive task practice. Figure 
13.1(b) shows that while mean performance markedly increases over 18 
hours of task practice, there is a slow rise in between-individual standard 
deviations in performance. From the initial to final practice sessions, standard 
deviations have changed from 3.19 to 4.49, an increase of 41 percent. 

The third example (Figure 13.1(c)) illustrates what happens when 
learners adopt different learning or performance strategies. The task for this 
example is a simple lookup task, where the learner is presented with nine 
pairs of nouns on the upper part of a computer display, and a test probe 
(which either has one of the matching pairs of words, or has two words that 
do not match) on the lower part of computer display (Ackerman & Woltz, 
1994), What happens in this task is that some individuals simply look up the 
words on each task trial, which is a strategy that minimizes effort on the 
individual trial level. We called these individuals “scanners.” Performing 
with this strategy rarely can be accomplished in less than about 1 sec/trial 
(1000 msec). Other individuals, however, work to memorize the word pairs 
while they are also looking up the words for each trial. Their efforts are 
greater at the individual trial level, but very quickly these individuals get 
quite a bit faster than their scanner counterparts, because they can retrieve the 
word pairs from memory much faster than it takes to scan the display. Thus, 
we Called these individuals “retrievers.” Retrieving the items from memory 
can be very fast, and expert retrievers performed about twice as fast as the 


best scanners (e.g. about 500 msec/trial). Eventually (i.e. after several 


hundred task trials), the scanners learn at least some of the word pairs, almost 
incidentally, and get faster at the task. 

When one looks at the overall performance and _inter-individual 
variability (Figure 13.1(c)), there is a general mean improvement in the speed 
of responding, as would be expected. However, there is an initial increase in 
between-individual variability as practice proceeds, since the retrievers are 
getting much better at task performance, and the scanners are profiting much 
less from each practice trial. Eventually, between-individual variability 
decreases, as even the scanners begin to memorize the word pairs. That is, an 
initial SD level of 421 ms increases to 521 ms (an increase of 24 percent) 
before declining to 406 ms. At the end of 1,350 task trials, there has been an 


overall decline in between-individual SD of only 4 percent. 


Restriction of Range 


When inter-individual variability declines with the development of expertise, 
a group of expert performers can be expected to show much smaller 
differences between them than do novices. The psychometric problem 
associated with such a restriction in range of performance is that correlations 
with measures of limited variability attenuate (i.e. they get closer to zero). 
This can make it very difficult to find tests that can predict individual 
differences in performance, simply because there is relatively little variance 
to account for by the predictor measures. Of course, this makes betting on the 
winner of competitive sports competitions a highly speculative activity, 
whether one is wagering money on the outcome of the competition or 
predicting the rank-order of individuals when validating an ability or 
personality test. In the final analysis, it is much easier to predict which 
individuals will develop expertise in a task that shows a Matthew effect with 
practice than it is to predict which individuals will develop expertise in a task 


that evidences a decline in inter-individual variability. 


Base Rate Issues 


In addition to restriction of range in performance, there is a more 
fundamental problem for developing valid predictors of performance, namely 
the problem of extreme base rates (i.e. the rate at which a behavior is 
exhibited in the population). It has been shown (e.g. Meehl & Rosen, 1955) 
that as a behavior becomes less likely to occur, such as when only 1 in 100 
college athletes ultimately end up playing professional sports, a test to predict 
the likelihood of reaching the professional teams must have extremely high 
validity to be practically useful. Thus, when expert performance on a task is a 
rare phenomenon, it may not be practically feasible to develop a selection 
measure that provides valid inferences for which individual is going to 
succeed at an early age. Instead, a more efficient and effective approach is to 
use a “multiple-hurdles” approach. For example, by the time a baseball player 
reaches a professional level, selection has occurred at entry into Little 
League, at entry into High School, and then perhaps at entry into a 
College/University team. Each of the shorter-term predictions is likely to be 
much more accurate than long-term predictions. Also, the base rates rise at 
each successive selection hurdle, yielding a higher a priori likelihood of 
successful selection. For an extensive discussion of this issue, see Ackerman 
(2014). 


Interdependence of Performance 


Another difficulty that arises in the study of expert performance in some 
tasks is that performance is not solely dependent on the efforts of the 
individual performer. In many occupations, ranging from sports (such as 
team efforts, or when there are other individual competitors, as in tennis or 
auto racing) to scientific discovery or technological research and 
development, performance success depends to a non-trivial degree on the 
actions or behaviors of others, or depends on environmental influences 
outside of the control of the performer. Thus, a baseball player’s batting 
performance is dependent on the skill level of the pitcher, perhaps nearly as 
much as it is on the skill of the batter. Or, the scientific contribution of a 
scientist may depend on how many other researchers are working toward the 
same goal — getting to the goal a few days or months ahead of the 
competition may signal the difference between fame and fortune on the one 
hand, and relative obscurity on the other hand (e.g. see the discussion by 


Watson, 2001, on the race to discover the structure of DNA). 


When expert performance is interdependent with the performance of 
others, the ideal measurement of an individual’s performance would be an 
average of multiple measures taken with as divergent a set of other 
performers as possible. For some types of expert performance, a round-robin 
type tournament would be one means toward accomplishing this goal; 
however, this kind of procedure is not practical in many different domains. 
Race car drivers do not compete in cars from all competitor manufacturers, 
football players cannot be assigned willy-nilly to different teams every week, 


and research professors cannot be easily moved around from one institution 


to another. When random assignment is not possible, more complicated 
Statistical designs are needed to attempt to disentangle the effects of the team 
or other performers on the performance of the individual. Sometimes, 
however, this is simply impossible to accomplish. Under these circumstances, 
the only acceptable solution is to create an artificial environment (such as a 
laboratory experiment with simulations) in which the _ individual’s 
performance can be evaluated in the absence of other performers. While these 
procedures can provide the needed experimental control, the risk is that the 
performance measurements taken under artificial laboratory conditions may 
not be valid representations of the actual real-world task (e.g. see Hoffman, 
1987). 


Trait Predictors of Expertise 


One of the most universal findings regarding individual differences in task 
performance over practice or education is that as the time between 
measurements increases, the correlations between measurements attenuates, 
though the correlations rarely drop all the way to zero. Sometimes, when the 
task is simple and skills are rapidly acquired, the decline in correlations 
between initial task performance and performance on later task trials is 


extremely rapid (e.g. see Ackerman, 1987; Fleishman & Hempel, 1955; 


Jones, 1962). When tasks are more complex, there is still a pattern of 
declining correlations, but it is much less steep. Intelligence test performance 
for children older than about age 5, for example, is very stable from one 
occasion to the next. Test-retest correlations with a lag of less than a year for 
an omnibus IQ test are in the neighborhood of 0.90. Correlations with a lag of 
a long time, such as age 6 to age 18, are indeed lower (r = 0.80 or so, see 
Honzik, MacFarlane, & Allen, 1948), but are still substantial. 

The important aspect of this general phenomenon (which is called a 
simplex-like effect, after Guttman, 1954; see Humphreys, 1960) is that when 
the correlations are low between initial task performance and performance 
after extensive practice, the determinants of initial task performance cannot 
be the same as the determinants of final task performance. The critical 
questions, from a psychometric perspective, are what are the trait predictors 
of initial task performance, what are the trait determinants of expert level 


performance, and what is the difference between the two? 


From the time of Immanuel Kant (1987/1790), philosophers and 
psychologists have referred to three major families of traits: cognitive, 
affective, and conative. Cognitive traits refer to abilities, such as intelligence, 
or domain-specific knowledge and skills. Affective traits refer to personality 
characteristics (such as impulsivity, conscientiousness, extroversion). 
Conative traits refer to motivation, interests, or more generally “will.” In 
addition, there are other traits that do not fit neatly into the tripartite 
breakdown, such as self-concept or self-efficacy. We briefly discuss these 
families of traits and their validity for predicting individual differences in 


expert performance, or in the development of expertise. 


Cognitive Traits 


Perhaps the most pervasive evidence for the validity of psychological 
measurements in predicting individual differences in the development of 
expertise is found for measures of cognitive or intellectual ability (e.g. see 


Jensen, 1998; Terman, 1926). Cognitive ability measures can be very general 


(such as IQ, or general intelligence); they can be broad (such as verbal, 
numerical, and spatial ability); or they can be quite specific (such as verbal 
fluency, computational math, or spatial visualization). From the first 


introduction of the modern test of intelligence (Binet & Simon, 1973), it has 


been clearly demonstrated that IQ measures can provide a highly reliable and 
highly valid indictor of academic success or failure. In fact, over the past 100 
years, IQ testing is probably the single most important application of 
psychological research in terms of application in the Western world. IQ tests 
have the highest validity for the purpose for which they were developed — 
namely, prediction of academic performance of children and adolescents (e.g. 


see Anastasi & Urbina, 1997). They provide significant and substantial 


predictive validity, but somewhat less so for predictions of adult academic 
and occupational performance. Narrower tests, such as verbal, numerical, and 
Spatial content abilities, when properly matched with the task to be predicted, 
can have somewhat higher validities for adults than general intelligence. 
However, the overall pattern found across many different investigations 
is that general and broad measures of cognitive/intellectual abilities are the 
most important predictors of performance early in training or learning. When 
tasks are within the capabilities of most performers, and declining inter- 


individual variance is observed, broad ability measures tend to show lower 


validity for predicting performance over task practice and instruction (e.g. see 
Ackerman, 1988; Barrett, Alexander, & Doverspike, 1992). That is, what 


appear to limit performance early in task practice (i.e. for novices) are the 


same abilities that are tapped by broad measures of intelligence, such as 
memory and reasoning. Individual differences in these abilities can determine 
how well a learner understands what is required in the task situation, and how 
effective the learner is in forming strategies and plans for task 
accomplishment. But, as we mentioned earlier in connection with inter- 
individual variability, a learner who quickly grasps the essence of the task has 
an advantage early in practice that diminishes as slower learners eventually 
begin to catch up over time. Skills like driving a car provide a good example 
of this kind of learning situation. Some learners grasp the procedures of 
scanning the various instruments and operating the controls quickly, and 
others more slowly, but after a few months of training and practice, the role 
of reasoning and memory in determining individual differences in 
performance is substantially diminished. 

There has been some evidence to suggest that when tasks are relatively 
simple and highly dependent on speed of perception and response 
coordination, there is an increase in the predictive validity of perceptual 
speed and psychomotor abilities for task performance as expertise is 
developed (e.g. see Ackerman, 1988, 1990; Ackerman & Cianciolo, 2000). 
That is, after extensive practice where most individuals become reasonably 
skilled at the task (such as driving a car or typing), performance is limited by 
more basic and narrow abilities (such as visual acuity and manual dexterity). 
Under these circumstances, the best ability predictors for individual 


differences in expert performance may be those measures that are associated 


with the limiting determinants of performance, rather than those abilities that 
are associated with reasoning and problem-solving. 

When attempting to select applicants for training or for job performance, 
one needs to take account of both the cognitive/intellectual correlations with 
initial task performance and the narrow ability correlations with performance 
after extensive practice (Beier & Ackerman, 2012). If training is a long, 
expensive process (such as training individuals to fly airplanes), it may make 
more sense to focus on using broad measures of cognitive/intellectual 
abilities for selection purposes, so as to minimize the number of trainees that 
wash-out of a training program. If training is less involved (such as in the 
selection of fast-food service workers or grocery-store checkout clerks), it 
may be more effective to the organization to base selection on perceptual 
speed and psychomotor measures, to maximize the number of expert 
performers in the long run. More elaborate selection procedures can be used, 
such as a “multiple-hurdle” approach. This procedure would provide tests of 
both cognitive and psychomotor measures, and applicants would be selected 
only if they pass a threshold score on both measures. Such a procedure 
maximizes both the likelihood of training success and the likelihood of high 
levels of expert job performance. 

When the tasks are not within the capabilities of many performers, or the 
task is highly cognitively demanding even after extensive task practice, 
general and broad content ability measures may maintain high levels of 
validity for predicting individual differences in performance long after 
training (e.g. for a discussion and examples, see Ackerman, 1992; Ackerman, 
Kanfer, & Goff, 1995), Most real-world jobs that are highly cognitively 
demanding have substantial domain knowledge prerequisites (e.g. the jobs of 


air traffic controller, neurosurgeon, software developer). One aspect that 


differentiates these tasks from other kinds of knowledge work are the strong 
demands of the tasks in handling novel information. Expert performance on 
these tasks is thus jointly influenced by individual differences in domain 
knowledge and by broad intellectual abilities (both general and content 
abilities, such as spatial abilities for air traffic controllers). 

Although domain knowledge can partly compensate for ability 
shortcomings when memory and reasoning abilities decline with age, world- 
class performance for such tasks generally remains the province of relatively 
younger adults (e.g. see Simonton, 1994). In contrast, for jobs that are 
predominantly associated with domain knowledge rather than the ability to 
deal with novelty, domain knowledge and skills appear to be relatively more 
influential than current levels of general and broad content abilities (e.g. see 
Chi, Glaser, & Rees, 1982). Such jobs include scientist, lawyer, radiologist, 
and so on. In such cases, the additional domain knowledge obtained through 
experience more than compensates for declines in general abilities with age, 
at least into middle age, and sometimes into early old age. Predictors of 
expert performance in these jobs appear to be those measures that tap the 
breadth and depth of relevant domain knowledge and skills (e.g. see 
Willingham, 1974). For a classification of job types along these lines, see 
Warr (1994). 

In general, across both motor-dependent tasks and knowledge or 
cognitive tasks, the key ingredient in maximizing the correlations between 
predictors and criteria is the concept of Brunswik Symmetry (Wittmann & 
Siig, 1999) — named after Brunswik’s Lens Model. That is, the content and 
especially the breadth of both predictor and criterion need to match. When a 
criterion is relatively narrow (e.g. specific task performance or a component 


of task performance), the best ability predictors will be those that are matched 


in both content (e.g. spatial, verbal, numerical, perceptual-motor) and breadth 
(in this example, a relatively narrow criterion would merit development of a 
relatively narrow ability battery for prediction purposes). Thus, predicting the 
typing speed of a typist is much more likely to be better predicted from a 
dexterity test (narrow) than an IQ test (broad). 


Affective Traits 


Affective, or personality, traits represent an area of great promise for 
prediction of the development and expression of expertise, but this area has 
little substantive evidence to date. Generally speaking, one can readily predict 
that serious affective psychopathology (e.g. schizophrenia, endogenous 
depression) is negatively correlated with development of expertise, ceteris 
paribus, simply because these patterns of personality are associated with the 
ability to manage oneself in society. It is noteworthy, though, that there are 
many counterexamples of experts who have had serious psychopathology, 
such as the Nobel Laureate mathematician, John Forbes Nash Jr., the Russian 
dancer Vaslav Nijinsky, Sir Isaac Newton, Robert Schumann, and many 
others. The unanswered, and perhaps unanswerable question, is whether these 
and other such individuals would have developed their respective levels of 
world-class expertise if they had not suffered from these affective disorders. 
In the realm of normal personality traits, one of the most promising 
constructs for predicting expertise is that of “need for Achievement” (nAch), 
proposed by Murray and colleagues (1938). McClelland and his colleagues 
(McClelland & Boyatzis, 1982; see Spangler, 1992 for a review), performed 


several studies that provided various degrees of validation for nAch in 
predicting successful performance in a variety of different occupations, but 
especially in the domain of managerial success. Other personality traits (e.g. 
Openness to experience, conscientiousness, extroversion) have been 
moderately linked to success in several different occupations (e.g. see meta- 
analyses by Barrick & Mount, 1991). However, in contrast to cognitive 


ability predictors of expertise, the direction and magnitude of personality trait 


measure correlations with success appear to be more highly dependent on the 
occupational context. That is, even when some ability traits are not directly 
relevant to a particular job or task, correlations between ability predictors and 
criterion performance measures are almost always positive, even if not 
particularly substantial. In contrast, with the exception of trait 
conscientiousness (i.e. dependability and achievement orientation) the 
relationship between personality and job or task performance depends on the 
job or task (Barrick & Mount, 1991). For example, extroversion may be 
reliably higher among experts in jobs that require interpersonal skills and 
leadership (e.g. politics, senior management), but the same trait may be 
relatively lower for experts in domains that require intensive individual 
efforts (e.g. mathematician, chess player). As a result, one perhaps might not 
expect that there will be particular personality traits that are associated with 


expertise across divergent domains. 


Conative Traits 


Some researchers have argued that nAch falls more in the domain of conation 
or will instead of personality, but this issue illustrates one of the more 
enduring issues in the field of personality research and theory — the problem 
of parsing the sphere of individual traits, when they do not really exist in 
isolation. nAch and many other conative traits, such as vocational interests, 
have clear and sometimes substantial overlap with personality traits (e.g. see 


Ackerman & Heggestad, 1997 for a review). 


In the 1950s, vocational psychology researchers converged on a set of 
core interest themes upon which individuals reliability differ (e.g. see 
Guilford, Christensen, Bond, Jr., & Sutton, 1954; Holland, 1959; Roe, 1956). 


Perhaps the most widely adopted framework from this research has been 


Holland’s “RIASEC” model — which is an acronym for six major vocational 
interest themes, namely: Realistic, Investigative, Artistic, Social, 
Enterprising, and Conventional (e.g. see Holland, 1997). It is possible to 
match these vocational interest themes with characteristics of jobs, so that 
individuals can be guided by vocational counselors to occupations that best 
match their underlying interests. It is possible that one could identify areas of 
expert performance within each of these different interest themes. There is a 
body of research that would support the notion that if individuals and jobs are 
matched on these themes, individuals are more likely to develop expertise 
(along with job satisfaction) than if there is a mismatch between the 
individual’s interests and the job characteristics (e.g. see Dawis & Lofquist, 


1984; Super, 1940). However, a match between the direction of interests and 


job characteristics is not in itself sufficient for predicting which individuals 
will develop expertise. 


The concept of “occupational level” (Holland, 1997), which represents 


how much challenge an individual desires in the task, is probably at least as 
important as the direction of interests is to the prediction of expertise. 
Occupational level is considered to represent a complex function of both an 
individual’s abilities and his/her self-concept, which is the individual’s 
estimation of his/her own abilities. There is probably more to this construct 
than self-concept and objective ability, in the sense that some individuals 
have both high aptitude for attaining expertise, and high estimation of their 
own aptitude, but lack the motivational drive to develop expertise. Kanfer 
(1987) has referred to this last component as the “utility of effort,” that is, the 


individual’s desired level of effort expenditures in a work context. 


Self-concept and self-efficacy 


Self-concept is a relatively broad set of constructs that parallel abilities (e.g. 
general intelligence, verbal, spatial, numerical abilities, etc.). Self-efficacy 
refers to task-specific confidence in one’s abilities to accomplish particular 
levels of performance (Bandura, 1977). From a Brunswik Symmetry 


perspective (Wittmann & Siif, 1999), predictions of expert performance from 


self-efficacy measures are likely to show higher criterion-related validity for 
performing specific tasks, mainly because of a closer match of breadth of 
predictor to breadth of criterion. For example, a self-efficacy measure might 
ask a golfer to provide a confidence estimate for making a specific putt, while 
a self-concept measure might ask the golfer to provide an estimate of his/her 


competence in putting, overall. There is also a motivational component to 


self-efficacy that entails what an individual “will do” in a task, in addition to 
what the individual “can do.” Existing data suggest that when the task is well 
defined, and when individuals have some experience with a task, self-efficacy 
measures can provide significant predictions of expert performance (e.g. 


Feltz, 1982). 


Communality among Predictors and Trait 
Complexes 


In terms of assessing and predicting individual differences in expertise, we 
have discussed how cognitive, affective, and conative traits all appear to play 
a role, at least to a greater or lesser extent. We would be remiss if we did not 
also note that while many researchers have only considered one or another of 
these trait families in predicting expertise, there is important shared variance 
among these traits. In terms of predictive validity, common variance between 
predictors means that their effects in a regression equation are not 
independent, and thus the total amount of variance accounted for in the 
criterion will ordinarily be less than would be obtained by adding the 
contributions of each trait family. Such common variance among trait 
families has even more important implications for theoretical considerations 
of the determinants of individual differences in expertise, in the sense that 
synergies across trait families may help us understand why individuals are 
oriented more toward some domains than others, or why some individuals 
succeed in developing expertise, while others only develop moderate or poor 


levels of task performance. 


Trait Complexes 


Ackerman and Heggestad (1997) reviewed the literature on the 
commonalities among abilities, personality, and interests. In the context of a 
meta-analysis, they found that there appeared to be at least four broad 
constellations of traits that often hang together, which they called “trait 
complexes” (after Snow’s concept of aptitude complexes; Snow, 1989). The 
underlying theoretical premise regarding these trait complexes is that they 
may represent configurations of traits that operate synergistically, in being 
either facilitative or impeding of the development of domain-specific 
knowledge, skills, and ultimately expert performance. The four trait 
complexes derived by Ackerman and Heggestad are shown in Figure 13.2, in 
a Spatial representation that overlays ability and personality traits with 
Holland’s hexagonal model of interests. The four traits were described as 


follows: 


The first trait complex shows no positive communality with ability 
measures, and is made up of a broad “Social” trait complex. It includes 
Social and Enterprising interests, along with Extroversion, Social 
Potency, and Well-Being personality traits. The remaining trait 
complexes do include ability traits. A “Clerical/Conventional” trait 
complex includes Perceptual Speed abilities, Conventional interests, and 
Control, Conscientiousness, and Traditionalism personality traits. The 
remaining trait complexes overlap to a degree, the third trait complex 
“Science/Math” is not positively associated substantially with any 
personality traits, but includes Visual Perception and Math Reasoning 


Abilities, and Realistic and Investigative interests. The last trait 


complex, “Intellectual/Cultural” includes abilities of Gc and Ideational 
Fluency, personality traits of Absorption, TIE [Typical Intellectual 
Engagement], and Openness, as well as Artistic and Investigative 


interests. 


(Ackerman & Heggestad, 1997, p. 238) 
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Figure 13.2 Trait complexes, including abilities, interests, and personality 
traits showing positive commonalities. Shown are: (1) Social, (2) 
Clerical/Conventional, (3) Science/Math, and (4) Intellectual/Cultural trait 
complexes (from Ackerman & Heggestad, 1997, copyright American 


Psychological Association. Reprinted with permission). 


These trait complexes lie at the heart of Ackerman’s (1996) investment 
theory of adult intellectual development. The theory, called PPIK, for 
intelligence-as-Process, Personality, Interests, and  Intelligence-as- 
Knowledge, along with a set of different outcome knowledge domains is 
illustrated in Figure 13.3. Briefly, the theory describes how individual 
investments of fluid intellectual abilities (processes like memory and 


reasoning) are guided by trait complexes that are facilitative (e.g. 


science/math and_ intellectual/cultural) and impeding (e.g. social) 
constellations of personality, self-concept, and interest traits. These 
investments, in turn, affect both the development of domain-specific 
knowledge related to expertise (such as science or humanities knowledge), 
and general crystallized abilities. In this framework, expert knowledge is 
obtained when there is a confluence of high intellectual abilities and high 
levels of affective and conative traits that are aligned with the particular 
knowledge domain. When abilities are moderate or low, but personality and 
interests are well aligned with the knowledge domain, some compensation is 
possible through investments of greater time and effort. However, even when 
suitable abilities are high for a particular domain, lower levels of matching 
personality and interests will likely tend to preclude development of expert 


levels of performance. 


Intellectual Personality/Interest/Self- Domain 
Abilities | Concept Trait Complexes Knowledge 


Humanities & 

xs syL_Social Sciences 
Humanities 

Current Events 


——-» Positive Influences Interpersonal 
knowledge 


asevenss p> Negative Influences 


Figure 13.3 Illustration of constructs and influences in the PPIK theory 
(Ackerman, 1996). Gf (fluid intelligence) represents “intelligence-as- 


process”; Gc = crystallized intelligence (from Ackerman et al., 2001, 


copyright American Psychological Association. Reprinted by permission). 


In several studies (e.g. Ackerman, 2000; Ackerman, Bowen, Beier, & 
Kanfer, 2001; Ackerman, Kanfer, & Beier, 2013; Ackerman & Rolfhus, 


1999) these trait complexes (and a few others) have been shown to be useful 


predictors of individual differences in domain knowledge among college 
students and middle-aged adults. Such results support the broader tenets of 
the PPIK investment approach, but they also show that the panoply of 
possible trait predictors across cognitive, affective, and conative variables 
could very well be reduced to a manageable set of complexes for practical 
predictive purposes. Although the trait complex approach has yet to be 
explored in terms of predicting individual differences in expertise within a 
single job classification or task performance, this approach appears to have 
promise both for improving understanding of what factors determine ultimate 
expert performance achievement, and for providing a small number of 
predictors that could be used diagnostically in expertise development 


contexts. 


Classification Issues 


One of the fields of psychometric applications that has been less explored 
outside of vocational counseling and large-scale selection (e.g. military 
placement) is the concept of “classification.” Whereas 
occupational/educational selection starts with a larger number of applicants 
than positions to fill, and focuses on which candidates will be the most likely 
to succeed, classification starts with the assumption that most, if not all of the 
applicants will be selected, and the goal is to match the applicant with the 
most suitable vocational/educational opportunities. Guidance counselors 
often try to operate in the classification context in that a major goal is to find 
the most suitable vocational path for each individual. From a psychometric 
perspective, attention is focused not specifically to level of ability, but rather 
to the pattern of strengths and weaknesses, so that the individual can 
effectively optimize the congruence of his/her characteristics and the relative 
demands of the occupation or educational opportunity. In this context, 
profiles of trait complexes have potential for predicting which educational 
and occupational opportunities will best match the individual’s relative 
strengths and weaknesses. 

Also, information about the individual’s knowledge structures (i.e. the 
patterns of domain-specific knowledge and skills that the individual has) can 
be used in the classification context, mainly because of the extensive body of 
research that has demonstrated that transfer of training from existing 
knowledge to new knowledge is more effective than novel learning. Thus, a 
psychometric approach to assessing the existing knowledge and skills of 


individuals might provide for more effective educational and vocational 


guidance, especially when this information is integrated with measurement of 
the cognitive/affective/conative trait complexes that indicate the individual’s 
dispositions toward or away from particular domains. Ultimately, the 
classification goal is to maximize the congruence between the individual’s 


characteristics and the characteristics of the job or educational program. 


Revisiting Intra-Individual Differences 


We have discussed several aspects of the psychometrics of expertise, with 
respect to inter-individual differences and a variety of intra-individual 
differences (e.g. learning, expert—novice differences). Although there has 
been extensive research in these areas, there is one aspect of intra-individual 
differences that has received relatively little attention in recent decades, that 
is, fluctuations of performance that are not related to learning or obvious 
external (e.g. environmental conditions) or internal sources (e.g. fatigue, 
illness). As we described earlier, traditional psychometric approaches to 
individual differences often relegate intra-individual differences to 
measurement error — something that reflects random fluctuations that are 
considered “noise,” where the trait measure is considered the “signal” of 
interest. However, as W. Somerset Maugham noted, “only the mediocre 
person is always at his best.” Whether it be a “hot hand” in basketball (see 
Bar-Eli, Avugos, & Raab, 2006), or a “batter’s slump” in baseball, there are 
numerous examples of experts (and non-experts) showing greater or lesser 
fluctuations in performance, even over the course of a single game. Some of 
these examples are indeed likely attributable to random fluctuations of 
external conditions, but others are possibly reflective of other individual 
characteristics or internal influences (e.g. distraction, worries, a tendency to 
“overthink” the task). Because study of these kinds of intra-individual 
fluctuations in performance can only be undertaken by intensive study of 
individual performance records, evaluation of these sources of variance are 


typically relegated to qualitative review, rather than quantitative evaluation. 


Yet, there remains much work to be done to better understand the sources of 
within-individual fluctuations in performance, even at the highest levels of 


expertise. 


Discussion and Challenges for Future 
Research 


In this chapter we have reviewed how psychometrics can play an important 
role in measuring the development of expertise and the prediction of 
individual differences in expert performance. Concepts of reliability and 
validity are central to all aspects of quantitative psychological research, but 
these concepts are too frequently implicit in experimental research, often to 
the detriment of the usefulness of the research. In the study of individual 
differences, reliability and validity are explicitly considered as integral to 
both theory and application. Special issues of measuring change and the 
problems associated with restriction of range in performance were reviewed, 
as these present challenges to many studies of expert performance. 

Over the past 100 years, there have been hundreds of studies that have 
focused on predicting individual differences in the performance of laboratory 
tasks, achievement in educational settings, and occupational performance, yet 
there are many areas, such as expertise in interpersonal interactions, that have 
relatively little research. We have only described a few illustrative examples 
of theory and empirical results from these investigations, as they relate to 
cognitive, affective, and conative traits. Two general sets of findings are 
noted below, along with a third domain of expertise that presents both 


opportunities and further challenges to theory and application, as follows: 


1. For tasks that require significant perceptual and motor components, 
most of the existing literature focuses on the effectiveness of ability 


predictors of individual differences in the development and expression 


of expert performance. General and broad cognitive abilities are most 
effective in predicting success with novel tasks, but perceptual and 
psychomotor abilities are often just as effective, if not more so, in 
predicting expert performance after extensive task practice. When the 
tasks are straightforward and accessible to most learners, cognitive 


abilities generally show lower predictive validity as expertise develops. 


2. For tasks that are predominantly based on domain knowledge and 
skills, we reviewed some of the findings for various trait predictors of 
expertise. It appears that a heuristically useful approach to 
understanding and predicting individual differences in the development 
and expression of expertise in domain knowledge tasks is one that 
focuses on the long-term investment of cognitive (intellectual) 
resources, through a small number of trait complexes (made up of 
cognitive, affective, and conative traits), leading to differences in the 
breadth and depth of domain knowledge and skills. Two trait complexes 
(science/math and intellectual/cultural) appear to be facilitative in the 
development of knowledge about different domains, whereas other trait 
complexes (e.g. social) may impede the development of traditional 
domains of expert knowledge (e.g. academic and occupational 


knowledge). 


3. In addition to these two types of expert performance, there is another 
one that has not received anywhere near the same level of attention — 


namely, expertise in interpersonal tasks. As noted by Hunt (1995), in the 


USA, there has been an increase in the number of jobs that are highly 
dependent on interpersonal skills — mostly in the service industries (e.g. 


childcare worker, customer service representative) — that has been 


concomitant with declines in manufacturing and traditionally blue-collar 
jobs. To date, there is too little available information on even how to 
identify and describe expert performance in this domain. We can 
speculate that there are affective and conative traits that may be effective 
predictors of expertise in this domain. There is both historical (e.g. 
Ferguson, 1952) and current research (e.g. Barrick & Mount, 1991; 
Motowidlo & Beier, 2010) that is consistent with this speculation. There 
is much more theory and research needed on the criterion side of the 
equation, along with a need for additional predictors on the 
cognitive/predictor side of the equation, before it will be possible to 


evaluate how well we can predict expertise in the interpersonal domain. 
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Introduction 


Experts’ performance should actually not be possible. The contrasts between 
lesions and healthy tissue in radiological images are barely visible, yet 
experienced radiologists only need a split second to notice that something is 
wrong. Chess grand masters regularly find the best solution, often after a 
single glance and without much deliberation, in an environment where there 
are allegedly more possibilities than there are atoms in the universe 
(Shannon, 1950). Similarly, returning a tennis serve of over 150 mph should 
not be humanly possible but the best tennis players regularly start their own 
counterattacks with the return. Given the fascination that experts’ 
performance usually elicits, it is not surprising that the scientific research on 
expertise is as old as scientific study of psychology (Binet, 1894). During that 
time there have been various methods of eliciting experts’ performance and 
studying its underlying mechanisms. Recent technological advances enabled 
researchers to have direct access to experts’ brains and tackle the question of 
the neural correlates behind experts’ performance. 

In this chapter we will review the research that shows how the brain 
accommodates the seemingly impossible feats of experts. We will first briefly 
consider the cognitive mechanisms behind experts’ outstanding performance, 
as they will help us understand the way in which the brain implements 
expertise. The main part of the chapter will focus on the neural 
implementation of expertise. We will follow the classical view of the human 
mind, which involves perception and the creation of a mental model of reality 


before acting. First we will deal with the neural changes associated with 


perceptual expertise domains, such as radiology, in which the main task is to 
rapidly detect relevant aspects of perceptual stimuli. In contrast, the crux of 
chess expertise is in the mental manipulation and simulation of the perceived 
stimuli. Chess experts are an example of cognitive expertise that we will 
review in the next section. Motor domains such as tennis feature the 
execution of movement sequences and will be dealt with in the section on 


motor expertise. 


Cognitive Mechanisms of Expertise 


Expertise is a prime example of how various cognitive processes, such as 
memory, attention, and perception, come together to enable a truly 
magnificent performance. Consider, for example, the opening example of 
skilled radiologists. Radiological images carry a wealth of information and 
one usually needs to carefully examine all of the tissue to make sure there are 
no abnormalities. Nevertheless, skilled radiologists can spot over 70 percent 
of pathologies even when the images are presented for only 200 milliseconds 
(Kundel & Nodine, 1975). The short presentation precludes the search 
process commonly associated with radiological skill and the radiologists have 
to rely on their first impression of the image to make an educated guess. 
Experienced radiologists are able to perform so well in so little time because 
they have stored numerous instances of radiological images in their long-term 
memory (LTM). These stored knowledge structures are called by different 
names, from schemas to scripts (see Gobet et al., 2001), and their primary 
function is to make sense of the incoming new stimuli. When expert 
radiologists examine radiological images, related knowledge structures are 
inevitably activated in their LTM. Even if there is no perfect match between 
the incoming image and the existing ones, the chances are that the vast 
knowledge base of radiologists enables them to quickly form a good working 
hypothesis of the incoming images. Less experienced radiologists lack this 
knowledge and cannot form a quick impression of the incoming stimulus. 
They are essentially guessing when they are forced to decide in a split 


second. 


The first impression based on the domain-specific knowledge stored in 
LTM has another consequence. It enables experts to attend to the most 
relevant aspects in the stimulus and disregard less informative features. The 
consequence is that expert radiologists need only a few fixations to locate the 


abnormality within a radiological image (Krupinski, 2000). Novices lack the 


domain-specific knowledge required to guide their attention and they 
consequently need to examine the stimulus in great detail, rarely 
distinguishing between more and less informative features. 

The interplay between domain-specific memory, attention, and 
perception is the core of the theories on radiological expertise (Kundel, 
Nodine, Conant, & Weinstein, 2007; Swensson, 1980). A similar mechanism 
is found in a cognitive domain — chess. As in radiology, the domain-specific 
knowledge stored in chess experts’ LTM enables them to quickly grasp the 
essence of chess positions. Experts not only have better recall of briefly 
presented positions from unknown games (Chase & Simon, 1973; de Groot, 
1978), but they are also faster at locating certain types of pieces in those 
positions (Bilali¢, Langner, Erb, & Grodd, 2010). Just as in radiology (see 
Reingold & Sheridan, 2011), experts’ strategies are vastly different from 
those of their less skilled peers — whereas novices need to examine every 
corner of the position to identify the pieces of interest, experts only need a 
few glances to perform the same task. The same efficiency is found when 
chess experts look for the best continuations. Experts do not necessarily look 
further ahead and simulate future situations in more depth than less skilled 
players, but they immediately examine solutions of high quality (de Groot, 
1978; see also Gobet & Charmess, Chapter 31, this volume). Their knowledge 
enables them to instantly orient themselves and concentrate on promising 


solutions to the problem at hand. Novices, on the other hand, do not have the 


luxury of the vast knowledge base that would guide their search. They need 
to examine numerous less promising paths before they stumble onto a 
promising solution. 

Motor expertise also features a similar mechanism. Tennis experts do 
not wait to see the ball in the air before initiating their motor response 
because no matter how fast they are, they will not be able to execute the 
movement in time (see Abernethy, Farrow, & Mann, Chapter 35, this 
volume). Instead, they anticipate where the ball is going to land by extracting 
the kinetic information from the player who is serving. Obviously, tennis 
experts would not be able to perceive the informative aspects of complex 
movements such as tennis serves if they did not have similar movements 
stored in their LTM. The stored movements enable experts to pick up 
important information in the serve motion, mentally simulate the most likely 
scenario, and predict where the ball is heading before it has even made 
contact with the racket. Tennis novices lack this domain-specific knowledge 
and have to wait for the ball’s trajectory to become apparent, when it is 
already too late to react. 

Despite the differences between perceptual, cognitive, and motor 
expertise domains, the cognitive mechanisms responsible for experts’ 
superior performance have been found to be remarkably similar. At the core 
of all types of expertise is domain-specific memory that has been acquired 
through prolonged exposure and practice within a domain of specialization. 
The content of memory may be different depending on the domain, a feature 
that is important to keep in mind when we talk about neural implementation 
of expertise. In all instances, however, the domain-specific knowledge 
enables quick orientation in a new situation by biasing the perceptual intake 


through guided attention. It is important to underline that the greater 


efficiency of experts is not caused by them performing more quickly the very 
Same strategies as novices. Instead, domain-specific knowledge enables 
experts to employ qualitatively different strategies that draw on their vast 
knowledge base. Experts’ strategies may be fast and even look effortless, but 
as we have seen, their efficiency is based on a complex interaction between 
numerous cognitive processes (Bilali¢ et al., 2010). As it turns out, this has a 


profound effect on the way the brain implements experts’ performance. 


How the Brain Accommodates Expertise 


One of the exciting technical developments in recent years is the availability 
of neuroimaging techniques for research. The most commonly employed 
neuroimaging technique is arguably functional magnetic resonance imaging 
(f{MRI), which will be featured in most of the research presented in the 
upcoming sections on functional brain changes associated with expertise. The 
structural brain changes, typically investigated with voxel based 
morphometry (VBM) for gray matter, and diffusion tensor imaging (DTI) for 


white matter, will be presented after the functional changes. 


Perceptual Expertise 


Functional Brain Changes in Perceptual Expertise 


When we talk about expertise, we usually have in mind highly specialized 
domains such as radiology. However, we are all experts in many aspects of 
everyday life as we can easily and efficiently find our way in the 
environment. For example, we are particularly skilled at recognizing objects 
in our environment, with perception of faces arguably the pinnacle of our 
perceptual skill (for a different perspective, see Ericsson, Krampe, & Tesch- 
R6mer, 1993). Most of us need just a second to recognize a face, a skill 
which we have honed our whole life. The instant perception of faces is a 
consequence of the way the faces are processed in our system. Instead of 
individually parsing face elements such as eyes, nose, and mouth, most of us 
can perceive a face in a single glance as a whole without paying attention to 
its individual components. This perceptual processing of individual elements 
as a Single unit is called holistic processing (Richler, Palmeri, & Gauthier, 
2012), which is the opposite of analytic processing where components are 
individually perceived before they are made into a whole. Since experts’ 
performance in many ways resembles holistic processing, the neural basis of 
face perception is of interest to expertise research. 

There are a number of brain areas involved in the different components 
of face processing (Duchaine & Yovel, 2015), but most researchers identify 
the fusiform face area (FFA), in the inferior side of the temporal lobe, as the 
main engine of holistic processing. Not only is the FFA more responsive to 
faces than any other stimuli, but damage to the brain tissue in and around this 
area leads to prosopagnosia, a condition in which people have difficulty in 
perceiving faces. People suffering from prosopagnosia fail to perceive a face 


as a whole and instead revert to analytical strategies of examining individual 


face components (Hadjikhani & de Gelder, 2002). The important finding here 
is that the FFA is more activated in experts than in novices when they 
observe objects within their domains of specialization (Bilali¢, Grottenthaler, 
Nagele, & Lindig, 2016; Bilalic, Langner, Ulrich, & Grodd, 2011; Ross et al., 
2015), which implies that the FFA is important for holistic processing, not 


just for perception of faces. 

A holistic-like process is typical of radiological expertise and one recent 
study (Harley et al., 2009) looked into its connection with the FFA. The 
activation levels in the FFA did not differ among differently skilled 
radiologists when they had to indicate if the cued part of a thorax X-ray 
contained a pulmonary nodule. However, the FFA was modulated by the 
performance, as the better the radiologists were at identifying nodules, the 
more they engaged the FFA. In contrast, the lateral occipital complex (LOC), 
the first complex station of the ventral stream that responds to objects and 
their shape (Grill-Spector, Kourtzi, & Kanwisher, 2001), was negatively 
associated with radiological performance, indicating that only the less skilled 
radiologists engaged the LOC. The engagement of the FFA and 
disengagement of the LOC as radiological expertise develops fits well with 
the notion that radiological expertise relies more on holistic and less on 
analytic processes. 

A recent study (Bilali¢ et al., 2016) extended these results by using a 
more sensitive multivariate voxel pattern analysis (MVPA). As can be seen in 
Figure 14.1, the activity patterns in and around the FFA of radiologists could 
reliably indicate whether the radiological images were presented or other 
neutral stimuli. There was no such differentiation in the FFA of medical 
students. The study further investigated one of the hallmarks of holistic 


processing, the inversion effect — experts perceive objects less well when 


they are inverted, that is upside-down, than when they are in their normal 
upright orientation (Thompson, 1980). The inverted stimuli impaired the 
performance of skilled radiologists, but it was also reflected in their FFA 
activity, which could be used to reliably indicate when upright and inverted 
X-rays were being presented. In contrast, the FFA in medical students could 


not differentiate between different orientations of X-rays. 
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Figure 14.1 Radiological expertise. The brain areas that can reliably 
differentiate between radiological images and neutral stimuli in 
radiologists (left) and medical students (right). 


For more details, see Bilali¢ et al. (2016). 
Another visual expertise that relies on processing wholes instead of 


individual parts is fingerprint matching. Unlike on TV shows where 


computers do all the work, in real life expert examiners are responsible for 


matching fingerprints from a crime scene with the fingerprints in a database. 
It turns out that fingerprints are perceived as a whole too (Busey & 
Vanderkolk, 2005). If holistic processing is disabled, for example by turning 
the stimuli upside-down, experts’ behavioral performance deteriorates. The 
struggle with inverted fingerprints has been reflected in the neural response 
as measured by electroencephalography (EEG). The negative amplitude 
produced from around the occipito-temporal areas, generally related to the 
FFA (Sadeh, Podlipsky, Zhdanov, & Yovel, 2010), reached a peak after 170 
milliseconds of the fingerprint presentation. In expert examiners, the 
abovementioned N170 component was more pronounced for inverted than for 
upright fingerprints, the characteristic pattern in the inversion effect (Rossion 
& Jacques, 2008). In contrast, the amplitudes were almost identical for 
upright and inverted fingerprints in students with little fingerprint experience. 

Visual expertise involves numerous cognitive processes, some of them 
unrelated to the FFA. A recent study (Harel, Gilaie-Dotan, Malach, & Bentin, 
2010) demonstrated that visual expertise is not only an automatic bottom-up 
process, but that it also engages other top-down processes such as attention 
and domain-specific memory. Once the attention of (car) experts has been 
engaged, together with the access to all their knowledge about the stimuli 
from their domain of expertise, the neural differences become evident, not 
only in the FFA, but all over the cortex including not only visual but also 
non-visual areas (for a review, see Harel, 2015). Expertise of any kind may 
involve many more cognitive processes than just the simple act of perception. 
Even face perception, a prime example of automatic holistic process, involves 
different aspects, from recognition of people, to judging their mood. 
Radiology is no different, and passive observing may also involve a number 


of other areas besides the FFA. For example, when radiologists look for 


inserted artificial modification in radiological images (blurred part of the 
image), they activate not only the middle and inferior temporal gyri but also 
the middle and superior frontal gyri (Haller & Radue, 2005), generally 
important for executive functions. Similarly, the left inferior frontal gyrus and 
the posterior cingulate gyrus, typically associated with reasoning and 
decision making, were more activated in skilled radiologists when they 
recognized pathologies within radiological images than when they had to 
recognize embedded animals and letters in the same images (Melo et al., 
2011). The sight-reading of music notes, another visual skill, also engages a 
number of areas besides the FFA (Wong & Gauthier, 2010). 

Besides the visual domains, auditory expertise is probably the most 
investigated perceptual expertise (see Lehman, Gruber, & Kopiez, Chapter 
28, this volume; Altenmiiller & Furuya, Chapter 29, this volume). Here we 
will review the expertise associated with other senses, such as the tactile, 
gustatory, and olfactory senses. One of the reasons why these kinds of 
perceptual expertise are not as often investigated as visual and auditory is that 
they rarely come alone, isolated from information from the other senses. 
People rarely need to orient themselves by only relying on touching, 
smelling, and tasting. That is why there are not many tactile experts. Blind 
people, however, do not have the luxury of using visual information and 
often rely on touch for information about their environment. They also tend 
to be better than their sighted counterparts on tactile tasks involving 
recognition and fine discrimination of two surfaces (for a review, see Kupers 
& Ptito, 2014). Neuroimaging studies (Stilla, Deshpande, LaConte, Hu, & 
Sathian, 2007; Stilla & Sathian, 2008) regularly show that blind people 


involve the areas that are normally reserved for visual processing, such as the 


LOC, the primary visual cortex (V1), and the fusiform gyrus. It is now clear 


that the activation of LOC in blind people during tactile tasks is a 
consequence of bottom-up processing — tactile information from the 
somatosensory cortex is being forwarded for further processing to bilateral 
visual areas (Lucan, Foxe, Gomez-Ramirez, Sathian, & Molholm, 2010). 
This direct connection between somatosensory areas and nominally visual 


areas is not available in sighted persons (Stilla et al., 2007; Stilla & Sathian, 


2008). In the case of sighted participants, the information from the 
somatosensory cortex goes to the right intraparietal sulcus (IPS). Indeed, the 
IPS predicts the performances on fine tactile discrimination tasks of people 
with intact vision but not those of blind people (Stilla et al., 2007). In 
contrast, the activation in the LOC, nominally a visual area, reliably predicts 
the tactile performance of blind people. 

As with tactile stimuli, gustatory stimuli rarely come alone. The flavor 
of our favorite dish is a combination of not only its smell and taste, but also 
its visual characteristics. That might be one of the reasons why people are not 
particularly good at identifying taste when other visual and olfactory 
information is not present. Yet some people compete in recognizing (and 
nicely verbalizing) the components of taste, for example wine experts. Wine 
expertise is probably exaggerated in wine tasting circles, as wine tasting is in 
many ways more an exercise in verbal description than perceptual skill 
(Morrot, Brochet, & Dubourdieu, 2001). Nevertheless, trained wine tasters 
are better able to distinguish the components in expensive wine than regular 
wine lovers lacking training (Goldstein et al., 2008). Neuroimaging studies 
on expertise in this domain are rare not only because of the dearth of 
gustatory experts, but also because of the difficulty of administering 
gustatory stimuli to participants inside neuroimaging devices such as MRI 


scanners. Two studies on wine experts (Castriota-Scanderbeg et al., 2005; 


Pazart, Comte, Magnin, Millot, & Moulin, 2014) indicated that the strategies 


of expert and novice wine tasters differ significantly. Experts analyze the 
taste components, since this is their main task, while naive participants 
perceive the taste in a global emotional manner. This is reflected in the neural 
implementation in which the expert wine tasters engaged the insula area, 
which extends antero-inferiorly into the caudal orbitofrontal cortex, as well as 
the dorsolateral prefrontal cortex (DLPFC). The first area is associated with 
the synthesis of gustatory and olfactory stimuli as well as with the judgment 
of taste quality, while the DLPFC is also probably related to the attention 
paid to the taste input and retrieval of verbal labels for the individual 
components. The analysis and identification of the taste components in 
experts inevitably activate the hippocampal and parahippocampal areas, 
which are responsible for memory processes. In contrast, novices engage 
more the amygdala, an area associated with pleasure and emotions. 

As with tactile and gustatory performance, olfactory experts are rare. 
Blind people, however, seem to reliably outperform sighted people on tasks 
that require olfactory discrimination (Rombaux et al., 2010). A recent 
neuroimaging study (Kupers et al., 2011) demonstrated enhanced activity in 
the primary olfactory areas (amygdala) and the secondary olfactory areas 
(orbitofrontal cortex and hippocampus) in people who lacked vision from 
birth, or lost it early in life, when they are compared to sighted participants 
on identifying odors. Another study (Renier et al., 2013) found that the 
activation in the posterior part of the right fusiform gyrus differentiated 
between skilled odor perceivers (mostly blind participants) and less skilled 
counterparts on categorization and discrimination of odors. Once the 
olfactory tasks were replaced by auditory tasks, the fusiform gyrus did not 


differentiate between the groups or their performance. 


In the next section, on structural differences, we will see that the 
piriform cortex is enlarged in perfumers, people who undergo training to 
become professional makers of perfumes. The only functional study (Plailly, 
Delon-Martin, & Royet, 2012) that has examined the brain activity of the 
perfumers during passive exposure to odors, found activation in the piriform 
cortex in both hemispheres all the way to the anterior part of the neighboring 


amygdala. 


Structural Brain Changes in Perceptual Expertise 


Expertise, involving seemingly impossible performances, puts considerable 
demands on the brain. The brain reacts to these new demands with structural 
and functional reorganization. An example of this reaction is the brain 
plasticity that we have seen in blind people, where the connections between 
somatosensory and nominally visual areas have been reestablished following 
the absence of visual stimuli. The structural brain changes as a consequence 
of perceptual expertise are arguably most obvious in music experts (see 
Altenmiiller & Furuya, Chapter 29, this volume). Here we will just briefly 
mention a few studies on the structural brain changes associated with 
olfactory expertise. 

The size of the olfactory bulb, the most rostral part of the brain, is 
positively correlated with scent identification (Buschhiiter et al., 2008; 
Rombaux et al., 2010). The cortical thickness of other areas important for 
olfactory perception, such as the medial orbitofrontal cortex and insula, was 
also. positively associated with performance on differentiating and 


categorizing smells (Frasnelli et al., 2010). The olfactory areas seem to be 


differently specialized, as the volume of the olfactory bulb predicted the 
identification of odor but the fine differentiation between two odors was best 
predicted by the amount of gray matter in the orbitofrontal cortex and 


piriform gyrus (Seubert, Freiherr, Frasnelli, Hummel, & Lundstrom, 2013). 


Cognitive Expertise 


The importance of connecting incoming stimuli with the existing knowledge 
structures in LTM is arguably most obvious in cognitive expertise. We will 
start with the neural changes in expert memorizers, then move to the neural 
differences between two kinds of calculators: abacus experts and mental 
calculators. We will finish the section with board games and spatial experts, 


who need to find their way in complex environments such as large cities. 


Functional Brain Changes in Cognitive Expertise 


The way to remember new information is to connect it with the already 
existing information in LTM. That is the basis of a mnemonic called method 
of loci, in which highly familiar locations along a route are used for storing 
information by making associations between the locations and the 
information being stored. When memorizers need to retrieve that information, 
they visit the well-known locations along the route in their LTM and find the 
stored information. In the first neuroimaging study featuring a number of 
exceptional memorizers (Maguire, Valentine, Wilding, & Kapur, 2003), a 
number of brain areas associated with spatial memory and navigation were 
more active during the encoding of information in memorizers than in the 
controls: the cerebellum, medial superior parietal gyrus, retrosplenial cortex 
(RSC), and posterior hippocampus. 


In another study (Raz et al., 2009), a person who is capable of reciting 


65,000 decimals of m using the method of loci was shown to activate only the 
medial frontal gyrus and partly the dorsolateral prefrontal cortex (DLPFC) 
when the retrieval of the first 500 digits of m was compared to the activity 
associated with counting to 100. When the same person was asked to encode 
a matrix of 100 digits presented in 10 rows of 10 digits, the activation was 
markedly different from that of the retrieval of highly learned material. 
Besides the visual association areas around the precuneus, lingual, and 
fusiform gyri, at the beginning of the process of encoding new material, the 
motor associative areas and insula were also active. Later during the encoding 
process, the activation in the visual associative areas disappeared while new 


activation in the DLPFC and the orbitofrontal cortex appeared. It seems that 


the memorizer employed visual associations at the beginning, which in turn 
activated visual and motor associate areas, but later when the memory load 
increased, the frontal areas responsible for working memory (DLPFC) were 
being employed. 

In a recent study (Yin, Lou, Fan, Wang, & Hu, 2015), another expert 
memorizer, who used a storytelling method, was compared to a control group 
on a task where a series of two digits had to be remembered. The memorizer 
had more activation during encoding of the digits in areas that are important 
for episodic memory: left superior parietal gyrus (SPG), left DLPFC, left 
premotor cortex, and bilateral frontal pole. In contrast, the controls, who 
employed a rehearsal strategy, had more activation in the areas typical for 
rehearsal (left middle and inferior frontal gyri) and in the occipital visual 
areas. A similar pattern of brain areas was involved when the recall phase 
was analyzed. As with other studies on expert memorizers, the memorizer in 
this study did not use working memory, which is reflected in the functional 
reorganization of his brain activity compared to control participants who do 
not employ mnemotechnics. 

Expert calculators, who can quickly manipulate numbers mentally, 
represent another group of cognitive experts. Here we report the study on a 
mental calculator R. Gamm (Pesenti et al., 2001), who can quickly execute 
complicated calculations such as multiplying two two-digit numbers within a 
few seconds (see also Butterworth, Chapter 32, this volume). Besides the 
typical brain areas responsible for the visual working memory that control 
participants engage when they perform calculations, Gamm additionally 
activated the areas typically associated with episodic LTM regions important 
for encoding and retrieving: the parahippocampal gyrus (PHG), medial 


frontal cortex, and upper anterior cingulate gyrus. The controls did not 


engage these areas during their mental calculations, which confirms that their 
strategies did not involve storing intermediate results of complex calculation 
in LTM for later retrieval, as was the case with R. Gamm. 

Abacus experts are a different kind of expert calculators because they 
employ an external device to deal with complex calculations. Unlike mental 
calculators who rely on LTM to store intermediate results and retrieve them 
later when they are needed, abacus experts use the visuospatial working 
memory they need for mental representation and manipulation of the abacus 
(Frank & Bammer, 2012). Even a simple digit-span task (Tanaka, Michimata, 
Kaminaga, Honda, & Sadato, 2002) elicited a marked difference in the brain 


activations between abacus experts and control non-experts. Non-experts 
engaged the left Broca and the neighboring ventral prefrontal cortex, 
responsible for rehearsal typical in the digit-span performance. The abacus 
experts engaged the superior and inferior part of the parietal lobe (IPL and 
SPL), visuospatial areas most likely involved in the generation of visuospatial 
images of the abacus, the manipulation of these, and the maintenance of 
abacus content. In addition, abacus experts engaged the upper dorsal parts of 
the premotor cortex (PMd), an area important for voluntary action and in this 
specific situation used for imagining the movements necessary for the 
manipulation of beads. The importance of PMd in abacus performance was 
confirmed in a case study of an abacus expert who lost her abacus skill after a 
stroke that also damaged her PMd (Tanaka et al., 2012). 

Another study (Hanakawa, Honda, Okada, Fukuyama, & Shibasaki, 
2003) employed a representative task of adding a series of numbers with 
abacus experts and found the same bilateral parietal-prefrontal network 
associated with their performance. Non-experts, on the other hand, activated 


predominantly the left frontoparietal areas responsible for rehearsal strategy. 


The study also featured a clever spatial control task, which required 
participants to mentally move an object on a grid. The task, not unlike the 
abacus task itself, produced essentially the same activations in both groups as 
the abacus task had elicited in abacus experts. Suddenly, even non-experts 
were using the right prefrontoparietal network. 

A recent study (Ku, Hong, Zhou, Bodner, & Zhou, 2012) measuring 
simultaneously {MRI and EEG activity indicated that at the very beginning of 
the abacus calculation (c.380 ms after stimulus presentation), experts engage 
the SPL and middle temporal gyrus. Later, around 440 ms, the PMd premotor 
areas become engaged in abacus experts. The study illustrates the temporal 
dissociation between the visuospatial and visuomotor processes in abacus 
expertise, as abacus experts need to build up their mental abacus before they 
Start to manipulate it. 

Board games also place a great burden on the brains of experts as they 
not only have to perceive numerous individual objects and retrieve their 
functions, but also mentally transform the situation in order to find the right 
path in a jungle of solutions. Here we will focus on the game of chess, which 
has often been used in research on expertise (see Gobet & Charness, Chapter 
31, this volume). We will demonstrate how the brain implements numerous 
advantages that experts have over novices, starting from relatively simple 
object perception and moving toward more complex pattern recognition and 
problem-solving processes. 

A recent series of studies (Bilali¢, Kiesel, Pohl, Erb, & Grodd, 2011; 
Bilali¢ et al., 2010; Bilali¢, Turella, Campitelli, Erb, & Grodd, 2012; Rennig, 
Bilalic, Huberle, Karnath, & Himmelbach, 2013) examined skilled object and 


pattern recognition in chess. Object recognition was investigated using a 3x3 


miniature board with only two pieces on it, as depicted in Figure 14.2. Chess 


experts were faster at identifying chess pieces as well as at retrieving their 
function and connecting them to other pieces. The eye-movement recordings 
showed that the experts’ advantage lies in greater familiarity with the pieces, 
as they do not have to fixate them directly to identify them. The advantage is 
even greater when they had to indicate whether two pieces form a check 
relationship, since, again, greater familiarity allows experts to encompass the 
whole constellation with a single glance. Figure 14.2 shows that there is a 
large overlap in the brain activation in experts and novices on such a simple 
check task. Many of these activated areas proved to be task-specific, as the 
same task involving the same board but geometrical shapes (e.g. square, 
diamond) instead of pieces, engaged many of the same areas. There were, 
however, also noticeable chess-specific task areas, and these areas were more 
activated in experts than novices: lateral temporal areas, including the 
bilateral posterior middle temporal gyrus (pMTG) and its neighboring 
occipito-temporal junction (OTJ), as well the bilateral supramarginal gyri 
(SMG). These areas were partly activated in novices in the left hemisphere, 


but only the skilled players engaged its right-hand counterparts. 
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Figure 14.2 Skilled object perception in chess. The brain areas activated 
when chess experts (middle panel) and novices (bottom panel) had to 
indicate if there is a chess piece on a miniature chess board (top panel). 


For more details, see Bilalic, Kiesel, et al. (2011). 


The other studies in the series employed the whole chessboard and 
chess-specific tasks of looking for certain types of pieces on the board and 
identifying threat relations between pieces. An additional manipulation was 
the randomization of chess positions, which tapped into pattern recognition 
processes. The same lateral areas as in the chess-specific object recognition 
were activated when experts were looking for specific pieces or threats: 
pMTG and SMG. These brain areas did not differentiate between normal and 


random positions, which is to be expected, as recognition of individual 


objects and their relations with other objects was present in both position 
types. The new activations were in the PHG, around the collateral sulcus 
(CoS), and the RSC. Besides being modulated by expertise, the PHG and 
RSC were sensitive to the randomization manipulation. Normal positions, 
where experts could use their domain-specific knowledge, engaged the PHG 
and RSC more than random positions, where pattern recognition was difficult 
if not impossible. Novices, on the other hand, did not have much activation in 
these areas when compared to other control tasks, and the position type did 
not play an important role. 

These areas were also found in other studies employing different 
paradigms. When the recall paradigm was used (Campitelli, Gobet, Head, 
Buckley, & Parker, 2007; Campitelli, Gobet, & Parker, 2005), experts had 
more activation in the vicinity of the PHG on the retrieval of normal positions 
than on random ones. The novices displayed the same level of activation. A 
recent study (Bartlett, Boggan, & Krawczyk, 2013) used simple observation 
of normal and random positions and identified the posterior cingulate, the 
Same region where the RSC is situated, as the region modulated by both 
expertise and randomization. 

The lateral areas (OMTG and SMG) are important for perception of 


objects and their functions, especially manmade objects such as tools 


(Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995). Chess pieces also 
possess characteristic functions rooted in movement, which may explain the 
activation of the dorsal stream area, SMG. The SMG is associated with 
explicit retrieval of the actions of tools. The distinct function of the pMTG 
and SMG may explain their involvement in the chess-specific task. The 
pMTG was activated both when a chess object was identified and when its 


function was retrieved, but the SMG was mainly engaged in the check task 


where the retrieval of the chess pieces’ function was necessary. The 
involvement of these two lateral brain areas was confirmed in the pattern 
recognition experiments where identification of pieces on the board full of 
other pieces did not activate the SMG, but the identification of threats did 
(Bilali¢, Kiesel, et al., 2011; Bilali¢ et al., 2012). The inferior (PHG) and 
medial (RSC) brain areas responsible for chess experts’ quick orientation 
based on pattern recognition, are associated with scene perception and spatial 
navigation, respectively (Epstein, 2008). These findings indicate that chess 
experts’ brains accommodate highly specialized cognitive processes. The 
PHG and RSC are important for forming an initial idea about the 
environment, based on matching of the incoming information with the stored 
chess patterns in LTM. Once the attention has been directed to the aspects of 
interest in the environment, the lateral areas (SMG and pMTG) take over for 
object identification and retrieval of their function. 

A number of studies on chess investigate how people decide on the next 
move, the essence of chess expertise. Unfortunately, many of them suffered 
from methodological problems where it is difficult to unpick the processes 
involved because of insufficient control (Amidzic, Riehle, Fehr, Wienbruch, 
& Elbert, 2001) or they simply did not investigate expert groups of 
participants (Atherton, Zhuang, Bart, Hu, & He, 2003; Nichelli et al., 1994). 


A recent study (Wan et al., 2011) used extensive control and expert shogi 


players to investigate the neural underpinning of the process of finding the 
best solution. First it was found that the similar areas involved in chess object 
and pattern recognition were also involved in shogi perception. The pMTG 
was engaged in both experts and novices, but the PHG and RSC were more 
activated in experts. The posterior cuneus at the medial side of the brain was 


also more engaged in experts, and this was also important for discrimination 


between normal and random positions in experts (the RSC and PHG were 
also sensitive to the randomization, but the differences did not quite reach the 
significance level). The posterior precuneus, together with premotor and 
motor areas, and the DLPF, was also engaged when both experts and novices 
had to find immediately a solution to a position presented for only a second. 
Experts, however, additionally engaged the head of the nucleus caudate, a 
part of the basal ganglia. The nucleus caudate did not respond in a number of 
control tasks, but its activation was positively associated with the 
performance of experts. 

One of the functions of the basal ganglia is formation and execution of 
the stimulus-response associations and their caudate parts are responsible for 
cognitive responses (Poldrack, Prabhakaran, Seger, & Gabrieli, 1999). It is 
therefore possible that recognition of the well-known positions is carried out 
in the precuneus, an area involved in imagery of visuospatial stimuli and 
episodic memory retrieval (Wagner, Shannon, Kahn, & Buckner, 2005), as 
well as in the PHG and RSC. The information is then sent to the nucleus 
caudatus and triggers typical responses to such constellations (Wan et al., 
2011, 2012). This possibility gains further credence when we consider that 
the precuneus is directly connected to the nucleus caudatus as well as that 
their activations fluctuate in a similar manner — the more activation at a 
particular point in time in precuneus, the more activation there is in the 
nucleus caudatus (Wan et al., 2011). 

An expertise of a more spatial nature is finding a way in a complex 
environment, a task at which taxi drivers excel. Similarly to chess players, 
taxi drivers have been used to investigate a number of issues. Here we will 
briefly review the functional studies that preceded the well-known structural 


studies we will consider in the next section. An early study on spatial 


expertise (Maguire, Frackowiak, & Frith, 1997) required taxi drivers and 


controls to find a particular destination using the shortest possible route. This 
topographical skill of taxi drivers engaged a number of areas in the medial 
temporal lobe: both hippocampi, both parahippocampal gyri (PHG), and both 
posterior parts of the cingulate, which we called the RSC (for a review, see 
Epstein, 2008). These areas were more activated in the typical topographical 


taxi-related activities than in other memory tasks. 


Structural Brain Changes in Cognitive Expertise 


The demands of cognitive expertise leave a profound effect on the functional 
organization of the brain. However, the study on exceptional memorizers 


(Maguire et al., 2003) could not find any structural differences between 


memorizers and the control group. In contrast, the expert who could recite 
60,000 digits of m (Raz et al., 2009) had more cortical mass than the controls 
matched for gender and age in the right anterior region of the cingulate gyrus, 
an area important for mentalizing and emotional processing. The expert 
employed the method of loci that was based on vivid and emotionally laden 
images, unlike the exceptional memorizers in the other study (Maguire et al., 


2003), which may explain the inconsistency in the findings. A recent study 


(Kalamangalam & Ellmore, 2014) measured the cortical thickness in Hindu 
Vedic priests, who are required to learn a large amount of information. The 
Vedic priests had a thicker left orbitofrontal cortex and right infero-temporal 
gyrus, the areas involved in LTM, than the matched control participants. 
Children who had more than three years of experience with the abacus 
have an overall higher degree of myelination than their peers with no 


experience with the abacus (Hu et al., 2011). This was particularly the case 


for motor and visuospatial pathways, which would mean that the information 
between these areas would be transferred more quickly in abacus than non- 
abacus children. Another intriguing finding is the difference in the corpus 
callosum, which connects the two brain hemispheres. Given that abacus 
experts engage both hemispheres for their calculation (Hanakawa et al., 


2003), abacus expertise may enhance the connections between the left and 


right hemispheres as a result of extensive experience with generation and 
manipulation of the mental abacus. 

Structural changes in board experts proved to be elusive. The described 
series of studies on skill object and pattern recognition (Bilalic, Kiesel, et al., 
2011; Bilalié et al., 2010, 2012; Rennig et al., 2013) could not established 
structural differences between experts and novices (Bilali¢, 2017). The 
situation was no different in shogi experts, where neither the training study 


(Wan et al., 2012), nor the study involving experts playing for decades (Wan 


et al., 2011), could find any significant differences. Other studies involving 
more participants were more successful, but the pattern of results remains 
inconsistent. A structural study on the experts of the board game Baduk (also 
called Go) found that their caudate nucleus was larger than in novices (Jung 
et al., 2013), but another study on a similar board game (Chinese chess), 
found precisely the opposite pattern — the nucleus caudatus was smaller in 
experts (Duan et al., 2012). Yet another study on chess expertise (Hanggi, 


Briitsch, Siegel, & Jancke, 2014) found no differences between the volume of 


the caudate nucleus in chess experts and that in matched controls who did not 
play chess. The same study found that the pMTG, an area implicated in 
skilled object recognition (Bilali¢, Kiesel, et al., 2011; Wan et al., 2011), was 
smaller in chess experts than in non-players. The cortical thickness was also 
decreased in the pMTG in experts but also in the SMG and the posterior 
precuneus, both areas involved in skilled perception of objects and patterns 
(Bilali€, Kiesel, et al., 2011; Bilali¢ et al., 2010, 2012). The connections of 


the brain also differ in chess experts compared to non-playing matched 


controls (Hdnggi et al., 2014). The superior longitudinal fasciculus, a 
pathway that connects the temporal lobe with the parietal and frontal lobes, 


was more dense and compact in chess experts. 


The smaller volume and thinner cortex are consequences of microscopic 
changes in neurons and their connections as the brain purges unnecessary 
connections. This does not necessarily mean that the brain activation in these 
areas as measured by fMRI should also be smaller (Lu et al., 2009), but it 
most likely means that the efficiency has been increased through the pruning. 
For example, the studies investigating the overall activation patterns in 
Chinese chess experts and novices (Duan et al., 2012, 2014) found that 
experts’ caudate nucleus was in much better synchrony with the infero- 
temporal and parietal areas implicated in skilled perception. 

Another domain where experts rely on their memory is orientation, as 
we find in the case of taxi drivers. An intriguing finding about the importance 
of the right hippocampus in spatial expertise from an early study (Maguire et 
al., 1997), paved the way for one of the most imposing series of studies on 
structural changes in expertise. In a now seminal study (Maguire et al., 2000), 
taxi drivers in London were found to have a larger posterior part of the right 
hippocampus than the control group. The volume of the posterior part was 
positively correlated with experience in the job. A later study (Maguire, 
Woollett, & Spiers, 2006) confirmed that bus drivers, who also had many 
years of driving experience, did not have a larger posterior hippocampus, thus 
ruling out driving as a possible cause of the structural change in the 
hippocampus. More direct evidence comes from a study that followed a 
group of people who enrolled in the official training program for becoming a 


taxi driver in London (Woollett & Maguire, 2011). Eventually, some of them 


obtained a taxi license after a couple of years of learning the locations of the 
London streets. When their post-training brain scan was compared to their 
pre-training scan, the posterior hippocampus was enlarged. The control 


group, who did not undergo the strenuous training, unsurprisingly did not 


display any change in the hippocampus. The really interesting question, 
however, is what happened to the trainees who started the program but 
dropped out. Again, their hippocampus did not show the change after they 
dropped out because they did not complete the training and acquire “the 
Knowledge,” as the London taxi license is often called. Their pre-training 
scans were not, however, in any other way significantly different from those 
of the future taxi drivers who completed the training. What was different was 
the amount of time the two groups invested in training: the trainees who 
became licensed taxi drivers practiced for 35 hours every week, those who 


dropped out spent only 17 hours per week studying for the examination. 


Motor Expertise 


As in other expertise domains, memory plays a crucial role in motor 
expertise. The nature of this memory, however, is different as motor domains 
contain kinetic information. The kinetic information is stored at different 
brain areas compared to the perceptual kind of information we have 
encountered in the previous sections. The kinetic information is then used to 
anticipate the course of action and prepare a response well in advance. The 
crucial neural component of motor expertise may be mirror neurons, a large 
subset of neurons located in premotor and parietal areas in primates 
(Rizzolatti & Craighero, 2004). Mirror neurons are responsible for 
movements but also become active if the same movements have only been 
observed in others. It may explain a range of different phenomena, but more 
importantly, it provides a plausible neural mechanism for experts’ 
anticipation skills. 

The following subsection provides an overview of the anticipation 
studies that involve evaluations of visual scenes within MRI scanners. This 
paradigm is by far the most common given the difficulty of executing actual 
movements in MRI scanners. Nevertheless, there are some neuroimaging 
studies that featured actual movements. These are covered in Altenmiiller and 


Furuya’s chapter on music (Chapter 29, this volume). 


Functional Brain Changes in Motor Expertise 


The human equivalent of mirror neurons is called the action observation 
network (AON), which encompasses premotor and parietal, and inferior 
frontal areas in humans (Turella, Wurm, Tucciarelli, & Lingnau, 2013). The 
most impressive series of studies on the AON in motor experts comes from 
dancing, a domain where a series of moves needs to be coordinated and 


executed. In the first studies (Calvo-Merino, 2004), ballet and capoeira 


experts and novices were shown videos of ballet and capoeira dancing. 
Experts had more activation than novices in the ventral premotor cortex and 
inferior parietal lobe, but the same areas also differentiated between the two 
groups of experts. Ballet dancers had more activation in the AON when they 
watched ballet than capoeira videos, while capoeira dancers’ AON reacted 
more in the videos from their own domain of specialization. Subsequent 
studies found that the premotor and parietal areas were more activated in 
modern (contemporary) dance experts when they watched actions that were a 
part of their movement repertoire than when they watched similar movements 
that they had not executed before (Cross, Hamilton, & Grafton, 2006). 

The importance of the motor component as compared to the visual 
component is demonstrated in a study which compared female and male 
ballet dancers during the observation of female and male ballet dancing 
(Calvo-Merino, Grézes, Glaser, Passingham, & Haggard, 2006). Both 
genders spent a large amount of time observing the actions of the opposite 
gender, but the AON was less sensitive to the observation of movements 
performed by the opposite gender. A recent longitudinal study (Cross, 
Kraemer, Hamilton, Kelley, & Grafton, 2009) confirmed that the AON seems 


to be primarily influenced by motor and not visual experience. A group that 
trained in a dance sequence only visually with video observation showed 
much less AON activation when observing a similar dance sequence than a 
group who received actual dance training involving movement. A similar 
conclusion is reached in a study in which basketball players were much better 
at predicting the fate of free throws than journalists and other expert watchers 
who had spent countless hours observing the game (Aglioti, Cesari, Romani, 
& Urgesi, 2008). 

Other studies investigated motor anticipation in fast-moving sports such 
as football, basketball, or racket sports. The first study was on tennis (Wright, 
Bishop, Jackson, & Abernethy, 2010) and indicated that besides the frontal 
and parietal areas, and parts of the AON, anticipation additionally activated 
the areas responsible for perception of body movements, the posterior middle 
and superior temporal lobe. The study on basketball (Abreu et al., 2012) 
involved experts and novices, who both activated the AON in a similar 
manner when they had to predict the direction of free throws. The differences 
were in the posterior temporo-occipital junction, also called the extrastriate 
body area (EBA) because of its involvement in perception of body parts 
(Downing, Jiang, Shuman, & Kanwisher, 2001). The same AON areas and 
temporo-occipital areas important for body perception are found to be critical 
for the anticipation skill in hockey experts (Olsson & Lundstr6ém, 2013). The 
subsequent studies on tennis (Balser et al., 2014a, 2014b) found significantly 
more activation in experts’ AON when they had to predict where different 
shots would land than in that of novices. As well as the AON, experts also 
activated the cerebellum more, and this area, together with the superior 
parietal lobe (SPL), successfully differentiated between successful (more 


activation) and unsuccessful predictions (less activation). 


It does seem that all three main parts of the AON, the prefrontal, motor, 
and parietal cortex, underpin experts’ anticipation. The parietal areas of the 
AON may, for example, integrate spatial information about different body 
parts. They can also receive information about the positioning of elements in 
the environment, such as players on a field. This information may then be 
sent to the (pre)motor cortex that is then simulating the most probable action 
that the person is going to perform. The information is most likely again 
exchanged with the parietal cortex, which in turn may exchange information 
with the prefrontal cortex to arrive at a possible course of action. Even the 
areas that are not nominally a part of the AON network, such as the pMTG, 
EBA, and cerebellum, may play a role in this interaction. The activated 
posterior MTG may send processed information about body motion to the 
parietal cortex. The cerebellum, on the other hand, is important for precise 
temporal sequencing of body movements, and, with its connection to both the 
motor and premotor areas, may provide another source for the premotor areas 
to simulate observed action. 

A number of studies featured temporal occlusion, a standard paradigm 
on investigating anticipatory skill in motor experts (see Abernethy, Farrow, 
& Mann, Chapter 35, this volume). It turns out that the anterior AON areas, 
the ventral and medial frontal areas as well as premotor areas, were activated 
more during early occlusion in tennis (Wright et al., 2010) and badminton 
experts (Wright, Bishop, Jackson, & Abernethy, 2011). In contrast, the AON 
areas at the back of the parietal lobe (SPL/IPL) were more activated in 
experts when they observed videos occluded late, than when watching the 
same videos occluded at an earlier point. Novices did not display this 
anterior—posterior differentiation between early and late occlusion activation. 


A helpful way to interpret these occlusion results would be to keep in mind 


that during the late occlusion the ball flight is available, whereas in the early 
occlusion there is no such information. The flight trajectory is most likely 
processed in the parietal areas, which would explain experts’ reliance on the 
posterior AON areas. In contrast, such spatial information is lacking in the 
early occlusion condition and experts are forced to make predictions based on 
the observed kinetic information. They simulate the motion themselves, 
which would explain the activation of the motor and premotor areas. The 
frontal areas would then become engaged for the final decision. 

If anticipation based on kinetic information is essential for 
understanding an opponent’s next step in sports, then deception, such as 
covering up body cues with a series of other less obvious ones, is also an 
important part of the repertoire of a motor expert. A couple of studies 
(Bishop, Wright, Jackson, & Abernethy, 2013; Wright, Bishop, Jackson, & 


Abernethy, 2013) that featured a deception condition, came to the conclusion 


that besides the AON, the discovery of deception in the first place involves 
the insula and the posterior cingulate cortex, areas normally important for 
social interaction (Grafton, 2009). The prediction of the consequences of 
deception, such as where the ball is going to land, engaged more the anterior 
cingulate cortex (ACC), known to play an essential part in conflict resolution 


(Carter & van Veen, 2007), and the nucleus caudatus. 


Structural Brain Changes in Motor Expertise 


Some of the most drastic structural brain changes are associated with motor 
expertise, such as music (see Altenmiiller & Furuya, Chapter 29, this 
volume). Studies involving other motor activities also found significant 
differences in experts’ brain structure. Basketball players, for example, have 
increased gray matter in parts of the cerebellum and striatum (Park et al., 
2009). Similarly, badminton players have a larger cerebellum as well as more 
pronounced connections between the frontal and parietal areas (Di et al., 
2012). Golf players, who also require skilled hand and arm movements, have 


enlarged premotor areas as well as motor-related regions in the frontal and 


parietal lobes (Jancke, Koeneke, Hoppe, Rominger, & Hdnggi, 2009). Other 
motor activities like karate require skilled coordination between arms and 


legs. This is reflected in an increased cerebellum and primary motor cortex 


among karate specialists (Roberts, Bain, Day, & Husain, 2013). In the studies 
of typists, the more experienced they were (and thus the better they were at 
typing), the more gray matter they had in the SMA, prefrontal cortex, as well 


as cerebellum (Cannonieri, Bonilha, Fernandes, Cendes, & Li, 2007). Power 


athletes (e.g. sprinters and jumpers), who require quick foot movement for 
their performance, have a larger anterior part of the cerebellum, an area 
important for regulating motor speed performance, than endurance athletes, 
who do not require such quick foot movement (Wenzel, Taubert, Ragert, 
Krug, & Villringer, 2014). 


Conclusions 


We have seen that expertise changes the functional and structural properties 
of the brain. In all cases there is a close connection between the cognitive 
mechanisms that enable experts’ outstanding performance and the way that 
the brain adapts to the cognitive demands of expertise. One of the main 
recurring themes is the importance of domain-specific knowledge stored in 
LTM for experts’ performance. The kind of knowledge that is employed for 
the performance dictates the brain areas involved in a particular kind of 
expertise. This may involve the infero-temporal areas in visual expertise, 
parietal areas in cognitive expertise, and (pre)frontal areas in motor expertise. 
The lack of domain-specific knowledge in novices is the main reason why 
their strategies are vastly different from those of experts who rely on such 
knowledge. Given the complexities associated with retrieval of this 
knowledge and its influence on other cognitive functions such as attention 
and perception, it is not surprising that associated neural changes in the brain 
are more extensive in experts than in novices. The demand and complexity 
that the brain is experiencing when implementing experts’ strategies may also 
explain the common bilateral activation patterns in many expertise studies. 
The areas needed for the execution of certain cognitive processes are usually 
lateralized to one hemisphere, but the brain recruits the same areas on the 
opposite hemisphere when it needs to share the sudden increase in the 
computational burden (Weissman & Banich, 2000). The characteristic 
engagement of the additional brain areas in the opposite hemisphere by 
experts is dubbed the double take of expertise (Bilali¢, 2016, 2017; Bilalic, 


Kiesel, et al., 2011; Bilalic et al., 2012). It is a hallmark of the neural 
implementation of expertise that reminds us that the brain changes associated 
with expertise are impossible to understand if the cognitive mechanisms 


involved in experts’ strategies are not taken into account. 
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Introduction 


Over the last four decades there has been increasing focus on understanding 
the nuances of exceptional development. Much of this work can be directly 


traced to Simon and Chase (1973; Chase & Simon, 1973; following de Groot, 


1965) who suggested that factors differentiating players of different skill in 
chess were the result of practice and experience rather than innate qualities. 
Since this seminal work, researchers in domains ranging from medical 
education (Wayne et al., 2006) and music (Ericsson, Krampe, & Tesch- 
Romer, 1993), to distance running (Young & Salmela, 2010) and basketball 
(Baker, Coté, & Abernethy, 2003b) have emphasized that key factors 
distinguishing experts from non-experts are largely the result of training. In 
sport, for example, the seemingly superhuman reactions of expert goaltenders 
in sports such as ice-hockey or soccer result from the use of advanced 
perceptual information to develop superior anticipation skills that are the 
result of training and experience (Mann, Williams, Ward, & Janelle, 2007). 
The notion that training and practice are important is largely uncontroversial, 
as noted elsewhere in this volume (see for example Ericsson, Chapter 38), 
and most models of skill acquisition and expertise development give 
significant weight to the role of practice and experience. 

The importance of training/practice in skill development serves to 
emphasize the value of collecting valid and reliable practice activity data. 
Unfortunately, accurate collection of this type of data over extended periods 
of time is not easy. In this chapter, we focus on common approaches and 


highlight some consistent issues associated with collecting data of this nature. 


This chapter builds on the excellent contribution “Time budgets, diaries, and 
analyses of concurrent practice activities” by Deakin, Coté, and Harvey from 
the first edition of the handbook (2006) but expands and updates this 
discussion. More specifically, we have included methods that are widely used 
in studies of expertise development to ascertain practice behaviors that both 
describe and delineate skilled performers/experts from their less skilled 
counterparts. Because our expertise is in the field of sport, we have focused 
our attention here, although we have expanded the discussion where possible 


to include other domains of expertise. 


Concurrent Approaches 


As the name suggests, concurrent approaches focus on the collection of 
practice data while it is happening (or as close to this time as possible). 
Although once restricted to the use of observational or diary methods, 
concurrent practice can be evaluated using a range of sophisticated methods. 
For instance, the increased availability of technologies such as Global 
Positioning Systems (GPS) has allowed sport researchers to determine total 
movement within a practice amongst individual athletes (Edgecomb & 
Norton, 2006) as well as tracking interactions between performers as part of a 
more complex system (Araujo, Silva, & Davids, 2015). Despite the increased 
sophistication of these methods, the approach to evaluating how individuals 
(e.g. students, athletes, musicians) spend their time remains largely the same. 
That is, some sort of time budget analysis is used to characterize (by minutes, 
hours, days, etc.) how the time period under examination has been spent (e.g. 
how much time was spent in possession of the ball in a soccer training 
game?). However, this time analysis can be coupled with kinematic measures 
of distance traveled or psychophysiological measures indicating work-rate, 
thus allowing for analysis of practice quality. To date, this type of practice 
analysis has not been used to assess skill development, likely because of the 
restricted access to technology to all but the most elite or professional 
performers. In addition, while this time budget approach has generally 
focused on the performer(s), it can be expanded to include family members or 
other significant agents (e.g. teacher, coach, instructor) whose interaction 


with the developing learner may be of interest. Time budget analyses used in 


previous research range from minute-to-minute evaluations of the micro- 
structure of practice such as Horton and colleagues’ (Horton, Baker, & 
Deakin, 2005) investigation of expert coaches, to hour-by-hour, day-by-day, 
or week-by-week evaluations of how experts structure their days (e.g. 
Ericsson et al., 1993; for a more thorough discussion of time budgets see 
Deakin et al., 2006). 

Although total time spent in various forms of activity and practice has 
been a staple of research in this area, more attention to the micro-structure of 
practice has been advocated (Deakin & Cobley, 2003; Hiittermann, 
Memmert, & Baker, 2014). The assumption of using metrics based on total 
practice time or even hours of practice is that the qualities that make up an 
hour of practice are similar across time and/or that all learners are equally 
engaged in the practice during the time under examination. Researchers have 
begun exploring this assumption in a variety of ways. Time motion analysis, a 
specific form of time budget analysis, is a technique widely used for profiling 
sport performance — mostly with respect to the physical demands, and mostly 
in a competitive context (rather than as a means to measure practice). In their 
recent review chapter, Carling and Bloomfield (2013) noted the range of 
contemporary systems used for collecting performance data, many of which 
could be used to supplement our knowledge about practice quantity and 
quality. Similar approaches have been used to investigate time use of expert 


coaches. More and colleagues (e.g. More & Cameron, 2014; More & Franks, 


1996, 2004), developed the Coach Assessment Instrument, to conduct 
notational analysis of coaches’ behaviors, taking a systematic time analysis 
approach to marking what coaches do when they coach. These approaches 


indirectly help to categorize what players are doing during practice. 


Diaries/Training Logs 
The use of diaries (e.g. Ericsson et al., 1993; Starkes, Deakin, Allard, 
Hodges, & Hayes, 1996) and training logs (e.g. Baker, Coté, & Deakin, 2005; 


Young & Salmela, 2010) is common in studies of expertise development for 


collecting or verifying practice data. Generally, these approaches allow the 
collection of the sequence and duration of different forms of activity over a 
period of time (e.g. day, week, month), typically dictated by the research 
question under investigation. In addition to objective data such as start and 
end times for each activity and/or intensity levels, subjective data such as 
ratings of enjoyment or mood can also be collected via training logs. An 
example of a simple one-day diary is provided in Figure 15.1. There are 
several advantages to these types of methods. First, training logs and diaries 
are often completed habitually (sometimes obsessively) by performers over 
years of training, thereby providing a ready-to-use longitudinal dataset on 
practice-related variables. Second, because diaries are so widely used by top 
performers, the introduction of an expanded log/diary (i.e. one that collects 
more variables to answer specific research questions) is not as disruptive as it 
might be to a non-user. Three notable disadvantages of this approach are (a) 
transcribing diaries into data can be time consuming, (b) performers’ diaries 
are typically completed sometimes hours after the training session occurs, 
which may affect accuracy of recall, and (c) diaries typically require 
individuals to complete intervals of a set length (e.g. 15 or 30 minutes), 
which may not be sensitive enough to capture the nuances of activity that can 
occur during the interval (e.g. a lot of different activities can occur during a 


30 minute interval). 
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Figure 15.1 Example of a time use diary showing an eight-hour block of 


time. 


Systematic Observation 


Another method widely used to collect practice data and other forms of 
behavioral information involves the systematic observation of practice while 


it is happening. This form of participant observation research (Atkinson & 


Hammersley, 1994) typically relies on a structured data collection form to 
collect specific forms of practice data to inform the research question(s) 
under investigation. For instance, in their evaluation of expert coaches, 
Horton et al. (2005) coded each specific utterance from expert coaches to 
determine the frequency of different forms of instructional behaviors in an 
attempt to better understand the nuances of coaching expertise. Observations 
of performers in their practice environments have the advantage of high 
external validity, although a disadvantage of this approach is that the 
randomness and unpredictability of some practice environments may interfere 
with a systematic approach to measurement (i.e. something might happen that 
is not accounted for in the data collection form). The use of comprehensive 
“field notes” is important for managing this concern. Another disadvantage of 
this approach is that performers may modify their behavior when they are 


aware of being observed (the so-called Hawthorne effect, e.g. Adair, 1984). 


Experimental Approaches 


One method of assessing practice among skilled performers that has not to 
date received much attention, is one where their practice decisions are 
assessed in what is known as self-determined practice. To date, this technique 
has been used in a sport-specific task (practicing target aiming kicking skills 
in Gaelic football; Coughlan, Williams, McRobert, & Ford, 2014) as well as 


in a task setting outside of the person’s domain expertise, that is, music 


experts practicing a Frisbee throwing skill (Hodges, Edwards, Luttin, & 
Bowcock, 2011). With this approach, the authors have studied the 
characteristics of practice when the performer (either a domain or non- 
domain expert) is in control of how to practice. Evidence has been presented 
in both studies showing that experts demonstrate a better quality practice, 
compared to their less skilled counterparts and that these effects are shown in 
both their typical domain as well as across task domains. In general, the 
experts showed evidence of practice scheduling that best adheres to best 
practice principles (e.g. going from low to high practice interference; 
requesting more feedback and instruction; or practicing skills that were 
weaker). This technique, if conducted in conjunction with systematic 
observation and questionnaires, has the potential to inform about the practice 
decisions and potential self-regulatory activities (within and between practice 
sessions) that lead to high levels of performance skill. 

In addition to these methods, the use of practice/training blogs might 
reflect an emerging methodological process with considerable relevance for 
ascertaining information about practice quality and quantity. For instance, the 


aspiring golfer Dan MacLaughlin had an extensive blog about his attempt to 


become an elite golfer, beginning with his first attempts swinging a golf club 
through to the acquisition of thousands of hours of deliberate practice 
(thedanplan.com, 2015). It is possible that the open-ended, and at times 
conversational nature of these blogs may provide useful data beyond that 


captured using more traditional approaches. 


Retrospective Approaches 


Retrospective approaches to collecting and assessing practice history data 
require participants to think back to the early years of their involvement in 
the domain of interest and reflect on past practice experiences, behaviors, and 
characteristics. Respondents typically recall their involvement in yearly, or 
sometimes two- to three-year intervals, with inductive estimates of 
intervening years. Information is then used to explore relationships between 
participation characteristics during the formative years and current/ultimate 
levels of performance. 

Retrospective approaches are popular among expertise researchers as 
they offer a practical, efficient alternative to the gold standard of 
developmental research — longitudinal studies. Below, we discuss three of the 
most common retrospective approaches to collecting and assessing practice 


history data. 


Questionnaires 


Given the extensive practice histories of experts, in many studies of long- 
term skill acquisition, questionnaire methods have become an increasingly 
useful tool to ascertain what activities have defined development and the 
relative and absolute time spent in these activities. Quantitatively oriented, 
retrospective questionnaires are popular in studies of expertise development 
across a range of domains, including music (e.g. Jabusch, Alpers, Kopiez, 
Vauth, & Altenmiiller, 2009), education (e.g. Kamp, Dolmans, van Berkel, & 
Schmidt, 2012; Loyens, Gijbels, Coertjens, & Codté, 2013), and sport (e.g. 
Bruce, Farrow, & Raynor, 2013; Hodges, Kerr, Starkes, & Nananidou, 2004). 


Within the sporting domain, the use of questionnaires was initially 
confined to simple investigations of hours spent participating in various 
activities (Helsen, Starkes, & Hodges, 1998; Hodges & Starkes, 1996; 


Starkes et al., 1996) and the participant samples involved in these studies 


were relatively small and homogeneous. More recently, however, several 
groups have administered in-depth questionnaires to large numbers of 
athletes, occasionally from a wide variety of sports, in order to obtain a more 
detailed understanding of the factors associated with sport expertise 
development (Gibbons, Hill, McConnell, Forster, & Moore, 2002; Memmert, 
Baker, & Bertsch, 2010; Moesch, Elbe, Hauge, & Wikman, 2011; Oldenziel, 
Gagné, & Gulbin, 2003; Ward, Hodges, Starkes, & Williams, 2007). 


Questionnaires are an attractive research tool as they offer both flexible 


and large-scale administration and analysis alternatives. While they may not 
elicit as much detail or context surrounding past experiences as qualitative 


interviews, and researcher control over response quality may be lower, the 


advantages afforded by questionnaires include the recruitment of large 
samples and efficiency in data collection and analysis. They can also be 
completed online relatively easily, which can help make them more attractive 
to complete and easier to analyze. These features often make questionnaires 
the preferred approach for the collection of practice history information. 
However, in addition to the limitations noted above regarding researcher 
control, the invariant approach used in questionnaires may lack the contextual 
flexibility to capture sport-specific nuances (e.g. the various forms and 


structures of different elements of practice). 


Qualitative Interviews 


Interviewing techniques comprise face-to-face discussions and hence enable a 
more individualistic and potentially deeper appreciation of expertise 
development. Normally, this approach involves some form of guided process, 
tracing the respondent’s history in the practice domain. Some researchers 
have suggested that this is a superior approach to collecting historical practice 
data as it allows for additional probing to obtain as much information as 
possible, minimize misunderstandings, and clarify any unexpected or 
ambiguous responses (e.g. Coté, Ericsson, & Law, 2005). 

In the field of sport expertise development, many researchers have 
adopted a qualitative, retrospective recall approach, utilizing open-ended 
interviews with athletes, parents, and coaches (Baker, Coté, & Abemethy, 
2003a; Baker et al., 2003b, 2005; Berry, Abernethy, & Coté, 2008; Bloom, 
1985; Carlson, 1988; Collins, Macnamara, & McCarthy, 2015; Cote, 1999; 
Durand-Bush & Salmela, 2002: Law, Cote, & Ericsson, 2007; Phillips, 
Davids, Renshaw, & Portus, 2010; Soberlak & Cote, 2003; Weissensteiner, 


Abernethy, & Farrow, 2009). Structured retrospective interviews have also 


been used in domains outside of sport, including music, medicine, 
mathematics, and science (e.g. Bloom 1985; Csikszentmihalyi, Rathunde, & 
Whalen, 1993; van de Wiel, Van den Bossche, Janssen, & Jossberger, 2011). 


In addition to more structured interviews, open-ended qualitative 


interviews have also been used to better understand the complex process of 
development and skill acquisition. For instance, Bruner, Munroe-Chandler, 
and Spink (2008) used a more general phenomenological approach to 


investigate the transition experiences of developing athletes making the 


transition into elite ice-hockey. The flexibility of this approach allows the 
interviewer to shift focus and direction as new issues emerge during the 
interview. 

Although qualitative interviews generate large amounts of detailed 
information, this approach is particularly labor intensive and time consuming. 
As a result, sample sizes are typically small, and often homogeneous. 
Therefore, the generalizability of findings beyond the respondent sample may 
be limited. Moreover, despite rigorous methods typically involved in 
thematic analysis of data, there is ultimately a degree of subjectivity that also 
impacts the generalizability, reproducibility, and validity of the findings. 
These methods might work best when used in combination with quantitative 
methods, or for exploration of new ideas and areas relevant to skill 


development (Firestone, 1987). 


Longitudinal and Prospective Follow-Up 


The ideal way to determine factors influencing skill development is to collect 
longitudinal data over a period of time that is thought to capture behaviors 
relevant to skill attainment. This method allows analyses of practice histories 
of successful and less (or non-)successful individuals without biases 
associated with assessing an already successful sample. This type of research 
affords high predictive validity because it allows some statements about 
causality (rather than just relationships). Because of the nature of longitudinal 
research, that is, it is time consuming, subject to high dropout, and difficult to 
perform when the aim is to include young individuals who are expected to 
make it to adult-elite levels, this technique is not commonly used. For this 
type of research to be most successful, large sample sizes are required, 
especially if the follow-up period is significantly removed from the initial 
period of data collection. 

In the skill development literature, there have been a number of attempts 
to perform longitudinal-type analyses. Probably the most common approach 
is the “cross-lagged” design, seen when researchers select two time periods, 
sampling from a set of successful performers at the youth level, and then 
following them up after a season/year (e.g. Vink, Raudsep, & Kais, 2015) or 
a few years (e.g. Hendry & Hodges, 2018). As with the methods outlined 
above, questionnaires have been the most common method of eliciting 
practice-related information and then attempts have been made to tie the 
types of practice to (i) later achievement, (ii) relative gains in achievement in 


task-related skills or selection to advanced/elite levels (e.g. Ford, Ward, 


Hodges, & Williams, 2009), or (iii) archival data where ratings or times are 
available to provide an index of skill (e.g. Campitelli & Gobet, 2008). 

In piano playing, Jabusch et al. (2009), followed up with individuals 
after ~2 years and showed that practice time was the only predictor of 
variation in “temporal evenness” in playing scales. In chess skill, Campitelli 
and Gobet (2008) were able to relate archival data about international chess 
skill ratings to group and individual practice amounts (see also Howard, 
2012). Finally, in sport, attainment of a professional contract in soccer has 
been related to early types of practice engaged during youth development 


(Ford et al., 2009). Although it is rare for individuals to collect practice data 


at multiple time points to see how these have changed over time and relate to 
skill-related metrics, this has been done in a few recent studies. In one study, 
high-performing youth athletes (~13 years of age) were followed up over a 
12-month period (Vink et al., 2015). Both motivation and practice increased 
in a linear fashion across the season. In another study, over 100 elite youth 
soccer players in the UK (aged between 13 and 15 years at initial data 
collection), were followed up over a period of 5 years, in 2.5 year intervals. 
The aim was to assess how well developmental soccer activities, ascertained 
retrospectively, predicted later success at adult professional levels (Hendry & 
Hodges, 2018). However, only nine succeeded to adult professional first team 
soccer, speaking to the high attrition associated with such longitudinal 
designs. More studies of this nature are needed to get a more reliable picture 
of skill development. The collection of performance times in sports, or 
rankings in domains such as chess, in concert with practice estimates is one 
relatively easy approach to collecting such longitudinal data (i.e. both are 
collected in a retrospective fashion, yet give an understanding of longitudinal 


changes within individuals). Importantly, optimal designs will include 


multiple assessments/time points to ensure that the dynamics of development 
and change, which are difficult to capture in cross-lagged designs, are 


captured. 


Issues with Collecting and Analyzing Practice 
Activity Data 


As with all research methods, it is important to be mindful of the limitations 
associated with the various approaches to collecting, analyzing, and 
interpreting practice activity information. Some of the major challenges faced 
by expertise researchers include establishing validity and reliability of data, 
identifying limits to recall, comparability and generalizability of findings, and 
extrapolation of descriptive findings to predictive application in practical 


scenarios. 


Determining Reliability and Validity 


A critical issue in the collection of practice data (particularly historical data) 
is establishing validity and reliability. Generally speaking, validity refers to 
the degree to which a method/instrument measures what it intends to measure 
(e.g. does a measure provide an indicator of deliberate practice or simply total 
time spent practicing?), while reliability refers to the “consistency” or 
“repeatability” of measures (i.e. does the instrument provide the same values 
from the same individual?). However, these concepts are much more nuanced 
than these simple descriptions suggest. When trying to determine the 
accuracy of training information, it is important to consider issues such as 
concurrent validity, convergent validity, test-retest reliability, and internal 
consistency. 

Concurrent validity is demonstrated when a test score or measurement 
instrument correlates well with a criterion measure such as another tool that 
has been previously validated; for example, cross-checking questionnaire 
data with diaries or structured interviews with a subset of the sample. 
Convergent validity indicates that measures designed to supposedly address 
the same underlying construct are, in fact, in agreement (Bryant, 2000). For 
example, this could relate to consistency between recall of practice activities 


collected from coaches, parents, and players (e.g. Baker et al., 2003b; Young 


& Starkes, 2006). Test-retest reliability refers to the ability of the instrument 


to elicit the same results when completed by the same respondent, under the 
Same conditions, on two separate occasions. This has been used when the 
Same questionnaire has been given to a subset of athletes on separate 


occasions (e.g. Ward et al., 2007). Internal consistency signifies that items 


within a single tool that propose to measure the same general construct, 
produce similar scores (Strube, 2000). It would also be valuable to consider 
the predictive validity of the measurement tool (i.e. the ability of the 
instrument to predict a criterion measure assessed at a later time point; 
Bryant, 2000). A more detailed discussion of predictive validity of practice 
history information is provided below, with reference to the practical 
application of findings arising from studies of expertise development. 

While most studies of expertise development typically give some 
mention of the need for validity and/or reliability to confirm the 
trustworthiness of the data reported, thorough examinations of validity and 
reliability of practice history information, particularly data collected via 
retrospective recall methods, are rare. Careful consideration must be given to 
the statistical methods utilized to assess reliability and validity of practice 
history information, along with the criteria adopted to interpret these analyses 
(see Hopwood, 2015 for a more detailed account of limitations associated 
with statistical approaches for analyzing reliability and validity of athlete 
training histories). Researchers are encouraged to use or develop tools with 
known, acceptable, levels of reliability and validity, and these values should 


always be reported. 


Limits to Recall 


Robust examinations of validity and reliability of practice history data 
contribute to an increased understanding of limits to human recall. Regardless 
of whether researchers use short-term approaches like time budgets and 
diaries, or long-term approaches such as retrospective questionnaires and 
interviews, there are limits to the type of data that can be accurately recalled. 
For instance, Coté et al. (2005) highlighted the limits of recall of subjective 
information during past training activities (such as whether activities 
performed years ago were fun or enjoyable). Trying to ascertain markers of 
“quality” practice rather than amount of practice continues to be a challenge 
with questionnaire methods, as does making statements about the equivalence 
of practice amounts across age, samples, and skill. One way this has been 
achieved is to ask performers to keep activity diaries that allow concurrent 
assessment of the amount and quality of practice (such as recording when a 
run was conducted, as well as how effortful or relevant it was to improving 


performance; e.g. Hodges et al., 2004). Assessment of practice has also been 


looked at as a function of time in season (so-called periodization). In this way 
it is possible to gauge practice amounts with respect to specific types of 
practice. For example, in athletics, there are typically pre-season, 


competition, and post-competition periods, which are defined by various 


levels of intensity in practice (e.g. Baker et al., 2005; Hodges, Augaitis, & 
Crocker, 2015). 

Summarizing findings of previous research in the sporting domain, 
Hopwood (2015) identified that athlete recall of sport-specific training 


history information in their main sport was typically acceptable; however, 


ability to recall details of involvement in other organized sporting activities 
was less reliable. Recall of participation in informal, unstructured sporting 
activities, often referred to as sporting play or deliberate play, appears to be 
particularly challenging, with reported reliability and validity for related 
measures proving to be significantly lower than structured practice activities 
(Hopwood, 2015). Acknowledgment of such limitations to recall is 


imperative during the design and interpretation phases of expertise research. 


Comparability and Generalizability of Findings 


While many researchers who study expertise development seek to explore 
similar questions surrounding identification of factors contributing to 
exceptional performance, to date there has not been a recognized gold 
standard measurement tool for the collection of practice history information. 
This issue is particularly evident in the authors’ domain of expertise (sport), 
but likely also applies to other fields of expertise research. In fields such as 
sport physiology and psychology, recognized instruments, methodologies, 
and scales are typically used to understand performance and measure 
intervention. Use of standardized methodologies not only promotes validity 
and reliability of measurement, but also allows for wide comparison of 
results and greater generalizability of findings. 

Generally speaking, studies of sport expertise development frequently 
incorporate unique measurement tools, designed specifically for the 
investigation at hand. While this allows researchers to tailor the investigation 
to their immediate needs, it also makes comparison of results and 
recommendations between studies difficult, because each study tends to ask 
athletes different questions in different ways. One notable example of this 
heterogeneity in methods comes in how deliberate practice is assessed. 
Researchers have distinguished activities primarily engaged for fun versus for 
improvement or activities that are self-led versus organized by an adult. 
However, depending on the research group, or sometimes on the study, these 
various activities have been defined as “practice” in one study, but “play” in 
another. Ericsson et al. (1993) originally specified individually determined 


practice activities (outside of formal classes in music) as deliberate practice. 


Yet, in research in sports, individual or self-led activities have been defined 
as a form of “play” and contrasted to organized/formal “practice” (e.g. Ford 
et al., 2009; Hendry, Crocker, & Hodges, 2014; Hiittermann et al., 2014; 
Ward et al., 2007). For people unaware of the details of each study, the 
mixing of methods, terms, and instruments causes considerable concern in 
interpretation and conclusions. A more unified approach to the definition, 
collection, and analysis of deliberate practice histories would be of 
considerable value to the study of expertise development, and warrants 


diligent attention and collaboration from leaders in expertise research. 


Descriptions Versus Prescriptions 


Another issue in collecting practice data from already elite performers is in 
determining how valuable the data are in terms of being of use to future 
performers. At any one point in time, practice data describe the structures and 
pathways in place that led to elite performance. In a different time period, the 
profile of an elite performer (and hence conclusions about processes) might 
look different due to changes in certain structures (such as the availability of 
elite youth academies, change in regulations governing amount of formal 
practice, change in rules regarding age grouping, etc.). Hence, descriptions of 
past experiences might not be applicable to current systems and therefore 
lack prescriptive value. It is important for researchers to be aware of changes 
in relevant systems, structures, and other contextual factors that may 
influence practice behaviors, and to provide basic commentary where 
appropriate. Another way to better understand whether the data are 
generalizable is to collect retrospective practice histories across a wide range 
of age-spans so that some cross-validation can be performed (e.g. Hendry et 
al., 2014; Ward et al., 2007). 

In addition to potential systemic changes that may affect athlete 
development, expert performance is known to be multidimensional, dynamic, 
highly individualized, and non-linear (Phillips et al., 2010). Thus, caution 
must also be exercised when extrapolating descriptive findings from studies 
of expertise development for the purposes of talent identification and 
selection among future performers. Practitioners must resist the temptation to 
predict future potential simply through comparison of young performers to 


their already elite counterparts. As described by Gulbin and Weissensteiner 


(2013), a multifactorial approach to talent identification, coupled with a 
carefully monitored talent confirmation period is advocated for a more 
functional approach to expertise development in practice environments. 

To date, limited attention has been devoted to examining the predictive 
ability of sport expertise studies, with most designs vulnerable to post hoc 
ergo propter hoc errors (i.e. since outcome Y followed activity X, outcome Y 
must have been caused by activity X). Understanding the predictive value of 
this research would be an interesting area for further study in order to 
maximize the impact of these studies in practical scenarios. The increasing 
number of longitudinal or prospective designs would be a method that would 
help address these issues of predictive ability. Throughout this chapter, it is 
clearly apparent that the various approaches to collection of practice history 
data each has strengths and weaknesses. Therefore, a multifaceted approach 
using strengths from various techniques should result in collection of data 
with the highest accuracy. For instance, in Baker et al.’s (2005) study of 
triathletes, a systematic interview process was used to collect retrospective 
data that were verified using day-to-day training logs from the athletes. There 
is also value associated with quasi-longitudinal methods of collecting data. 
Sampling across different age groups in a cross-sectional manner enables 
more reliable recall of practice, given that the performer is not required to 
retrospect so far in the past. As well, this method of gaining practice histories 
across multiple age groups concurrently allows for more valid conclusions 
(based on the degree of parity) about the process of development across 
multiple samples of potential experts. However, there are issues associated 
with assessing practice in young samples (e.g. 8-year-olds) and in knowing 


whether their practice trajectories represent those of adult experts, given the 


high likelihood that only a few will excel at adult levels (for an example of 
this method see Ward et al., 2007). 


Concluding Thoughts 


The collection of accurate and reliable practice data will remain a staple of 
studies of expertise development and skill acquisition. As we have noted in 
this chapter, there is some range in the methods used to collect these types of 
data, with variation in techniques often, but not always, accompanying the 
types of research questions investigated. Despite the long history of use with 
many of these methods, issues of reliability and validity remain. Conducting 
regular and thorough reviews and evaluations of the reliability and validity of 
the various methods used in expertise methods is important for determining 
the appropriateness of different methods. 

There are also still relatively few studies where longitudinal or 
prospective techniques have been adopted to study skill progressions. This is 
perhaps not surprising given the logistical constraints of these approaches; 
however, this may soon change. Technological advances including 
widespread use of mobile devices, readily created research-driven “apps” and 
ubiquitous Internet access allow researchers to have greater data collection 
power than ever before. Online methods of data collection and verification, 
including, for example, the use of daily training logs coupled with GPS 
tracking and/or objective physiological measures of workload may lead to 
continued improvements in the quality of practice data. These improvements 
could ultimately lead to significant developments in our understanding of 


what it takes to become an expert. 


Author Note 


Thanks to Dr. Brad Young and Dr. Nick Wattie for helpful comments on 


earlier versions of this chapter. 
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Introduction 


Studying the acquisition and maintenance of expertise can be done by 
applying longitudinal group studies. In a longitudinal study, multiple 
measurements are taken within each participant, monitoring the development 
of their current performance toward peak performance in the future (i.e. 
expert performance), in relation to possible underlying mechanisms (i.e. 
performance characteristics) over time. This type of study can be 
characterized as observational. This means that the environment is not 
manipulated when data are gathered. Because of the observational nature, it 
has been argued that these studies may have less power to detect causal 
relationships than experiments, which are extremely hard to organize in 
practice not only because of practical challenges but also because of ethical 
considerations. In a longitudinal study, several observations of the same 
participants are conducted over an extended period of time, sometimes lasting 
many years. Since there are repeated observations at the individual level, 
longitudinal studies are highly relevant and have more power than cross- 
sectional observational studies, by virtue of being able to exclude time- 
invariant unobserved individual differences, and by virtue of observing the 
temporal order of events. The benefit of longitudinal studies is that it is 
possible to detect developments or changes in the characteristics of the target 
population at both the group and the individual level. The key here is that 
longitudinal studies extend beyond a single moment in time, resulting in 
establishing sequences of events. As such, the same variables are observed 


repeatedly over long periods of time and can be correlated to an individual’s 


current level of performance but also prospectively with expert level of 
performance in the future. On the other hand, some of the disadvantages of 
longitudinal studies include the fact that they take a lot of time and are very 
expensive, making them not very feasible. Also, it is hard to repeatedly 
measure the same individual and obtain an equal number of measurements 
for each participant, and high dropout rates from a longitudinal study are no 
exception. Therefore, the value of cross-sectional studies preceding a 
longitudinal study should not be underestimated. A researcher planning a 
longitudinal study might start the research with several cross-sectional studies 
in order to have some idea on performance characteristics that can best be 
used to discriminate between individuals with different levels of 
performance, instead of immediately spending a lot of time, energy, and 
money on a complex study design with possibly some “wrong,” redundant, or 
non-discriminating measures. 

When studying the acquisition and maintenance of expertise, one of the 
first challenges is how to define expert level of performance and unravel 
which variables are underlying performance, the so-called performance 
characteristics. A second challenge is how to measure performance and its 
performance characteristics, and how to study the development thereof. A 
third challenge concerns the choice for the most appropriate study methods 
and statistical analyses. In the current chapter, we will explore these 
challenges by starting from the Differentiated Model of Giftedness and Talent 
(Gagné, 2010). This model distinguishes between natural abilities in both the 
physical and mental domains on the one hand, and competencies in a variety 
of fields on the other, e.g. academic, technical, science and technology, arts, 
social service, administration/sales, business operations, games, and sports 


and athletics. Since the choice for study methods and statistical analyses is 


closely related to, or even intertwined with, the content of that study, i.e. 
expertise in a certain field, the focus in this chapter will be on the field of 
sports while discussing the aforementioned challenges. However, the line of 


reasoning can be applied to other fields of expertise as well. 


First Challenge: Defining Expertise 


Expert performance in sports is defined as the consistent superior athletic 
performance over an extended period (Starkes, 1993). There is a clear, 
inverted U-relation between age and level of performance (see Figure 16.1). 
In youth competitions, the older athletes (e.g. 15-year-olds) usually 
outperform the younger athletes (e.g. 10-year-olds). Looking at the 
development of performance over time within an athlete, from a young age 
on, the level of performance increases. Typically, a youth athlete’s 
performance improves with age. This is related to an improvement of 
multidimensional performance characteristics, such as anthropometrics, 
physiological, technical, tactical, and psychological skills (Elferink-Gemser, 
Visscher, Lemmink, & Mulder, 2004, 2007). These characteristics are 
influenced by maturation, learning, and training (Elferink-Gemser & 
Visscher, 2012). 


Expert performance 


Performance 


Age 


Figure 16.1 Hypothetical development of performance toward expert 


performance through lifespan: an inverted U-curve. Note: for reasons of 


clarity, the relation between performance and age is illustrated linearly. In 
reality, this will be a non-linear relation showing huge inter-individual 
differences concerming accelerations as well as decelerations in 
performance developments over time, in many cases also including periods 


of temporary decrease in performance. 


Acquisition of Expertise 


Not all fields of expertise share the same age at which expert performances 
are delivered. This is also the case within fields, such as in sports (see Figure 
16.1). Differences in age coincide partly with the age at which athletes are 
considered to be ready to compete in the senior competitions. For example, 
the age of transition to senior competition for girls in gymnastics and 
swimming is 16 whereas in field hockey, judo, soccer, speed skating, tennis, 
and volleyball this is 18, and in handball the age is 20. For some sports, there 
is even a difference between girls and boys concerning the age of transition 
with boys moving to the senior competition at a later age than girls. For 
example, boys compete in senior gymnastics and swimming competitions 
from the age of 18 (girls 16), and in handball from the age of 21 (girls 20). In 
cycling, there is an espoirs competition up to the age of 23 after which the 
athletes move to senior competition. However, if the level of performance of 
a youth cyclist already equals or exceeds that of top senior competition level, 
the cyclist is allowed to compete in that competition before the age of 23. 
When studying the acquisition and maintenance of expertise, one has to 
take the age of the participants in relation to the expected age of peak 
performance into account. Research shows a trend of peak performance at 


increasing age in endurance sports (Schulz & Curnow, 1988; Lepers, Riist, 


Stapley, & Knechtle, 2010), ranging from peaking at around 20 years in the 
middle distance events (Swimming, around 2—15 min) to peaking at around 
39 years in long duration events (ultra-distance cycling, around 27—29 h) for 
both males and females (Allen & Hopkins, 2015). In Figure 16.1, this is 


illustrated by the various inverted U-shapes. One explanation may be that 


physical performance characteristics important for success in endurance 
sports generally increase progressively with increasing training histories and 
thus age (Zaryski & Smith, 2005). Most physical capacities of humans tend 
to peak around the age of 30 (Gabbard, 2004). 


Nevertheless, it seems possible to reach peak performance in endurance 


events at either much younger (i.e. middle distance swimming, speed skating) 
or older ages (i.e. ironman triathlon), implying that factors other than 
physiology contribute to performance. Older athletes may continue 
progressing by accumulating improvements in cognitive capacities that offset 
the inevitable plateau in physical ability (Allen & Hopkins, 2015). This is 
supported by research on job performance showing the human capacity to 
continue accumulating crystallized intelligence (e.g. knowledge, experience) 
until the age of at least 60 years (Salthouse, 2012). Also in younger 
endurance athletes, psychological performance characteristics seem to be 
essential for success in addition to the physiological requirements 
(Sokolovas, 2006; Allen, Vandenbogaerde, & Hopkins, 2014; Wolfrum, 


Knechtle, Riist, Rosemann, & Lepers, 2013). To define expertise is even 


harder when there is not a single objective performance criterion, such as a 
time or a distance. There is a clear difference between sports with and 
without an objective criterion for each participant (Giillich & Emrich, 2014). 
Examples of the first are athletics, swimming, ice speed skating, and weight 
lifting in which individual performance is measured in centimeters, grams, or 
seconds and the task is to minimize time or to maximize distance or weight. 
Examples of the latter are game sports such as field hockey and soccer where 
team performance is measured in the number of goals made and is influenced 
by the performance of the opponent team. An athlete’s performance is rated 


subjectively by others, such as scouts, coaches, and trainers. For example in 


soccer it is difficult to decide who is the better player, the attacker who scores 
the goal, or the one who passes the assist, the midfielder who organizes the 
team or the defender who prevents the opponents coming near the goal. (Not 
to speak of the goalkeeper.) In many other fields than sports, it is also quite 
hard to objectively define expertise because of different tasks and changing 


circumstances under which to perform. 


Maintaining Expertise 


Not only the age at which an athlete reaches expert performance (time needed 
for the acquisition of expertise) but also how long he or she is able to perform 
at the expert level of performance (maintaining expertise) differs between 
athletes, and as such also has consequences for the choice of the study 
population. For example, for some athletes expert performance is reached 
“only” one competitive season or even only during one single competition, 
whereas for others the period of expert performance is repeated over several 
Olympic cycles, maintaining expertise. This has consequences for the study 
design. In addition, it has to be noted that in most sports to maintain expertise 
it is not sufficient to maintain performance. The potential difference between 
maintaining performance and maintaining expertise needs to be taken into 
account when studying expertise. Any performance, including expert 
performance, is to be placed within its context. For example, the 2004 
Olympic champion swimming the 100 m freestyle ended in fifth place in the 
2008 Olympic Games, although he was faster in 2008 compared to 2004. It 
turned out that his competitors improved themselves even more than he did in 
the four years between both Olympic Games. As such, also at the top, an 
athlete continuously has to improve his performance to be able to compete at 
the highest level of performance compared to other world-class athletes. This 
is reflected in the improvement of world records in many types of sports. In 
other words, sports in themselves evolve as well with increasingly higher 
standards of expertise to meet in order to win the next Olympic gold medal or 
the World Cup (Elferink-Gemser, Huijgen, Lemmink, & Visscher, 2012). As 


a consequence, when studying expertise, performance needs to be placed 
within context at all times. 

Regardless of how long the athlete maintains expertise (i.e. performs at 
the expert level compared to other competitors), inevitably his level of 
performance will drop at some time while getting older because of the aging 
process. In other words, the window of opportunity to deliver expert 
performance is limited. At older ages, the opposite relation between age and 
level of performance becomes clear. This is also shown in Figure 16.1. In the 
master competitions, the younger athletes (e.g. 50-year-olds) usually perform 
better than the older athletes (e.g. 60-year-olds). However, when looking at 
the underlying performance characteristics in more detail, it appears that not 
all characteristics decline at the same rate. For example, in contrast to motor 
performance, skilled perception appears resistant to age-related declines over 
time in handball goalkeepers through the use of compensatory mechanisms 
(Schorer & Baker, 2009). In line with findings in general populations that 
speeded performance declines with age, Fischer and colleagues (2016) show 
that handball coaches are able to maintain accuracy skills better than reaction 
timed tasks. Another study showed that expertise in a perceptual-motor task, 
such as the basketball free throw, can be retained in older athletes (Fischer et 
al., 2015). Also other characteristics, such as relative age, playing position, 
and performance are suggested to affect career length in professional sports 
(Baker, Koz, Kungl, Fraser-Thomas, & Schorer, 2013; Steingréver, Baker, 
Wattie, & Schorer, 2016). 


Second Challenge: Studying (the 
Development of) Performance and 
Performance Characteristics 


Specific Performance Characteristics 


Although in sports current performance level is still the number one measure 
for selection teams, the relevance to take into account factors underlying 
performance level, the performance characteristics, and the development 
thereof is high when it comes to youth selections. As such, it is recommended 
to include not only measures of performance in studies on acquiring 
expertise, but variables underlying performance as well. Dependent on the 
requirements of the task at hand (e.g. shooting the ball in soccer or 
accelerating on ice in speed skating), an athlete needs a certain combination 
of performance characteristics to perform well (Elferink-Gemser, Jordet, 
Coelho-e-Silva, & Visscher, 2011). To monitor development, these 
performance characteristics can be measured at specific moments in time; for 
example at the start of a competitive season and near the end of it applying a 
longitudinal study design. 

To a certain extent, compensation between performance characteristics 


is possible (Vaeyens, Lenoir, Williams, & Philippaerts, 2008). An example is 


the observation of the interval endurance capacity, motivation, and the 
tactical skills of high-performing youth field hockey players (Elferink- 
Gemser, Visscher, van Duijn, & Lemmink, 2006). All these players were 
selected for a prestigious talent development program of a professional field 
hockey club in the Netherlands and were measured regularly by way of field 
tests and questionnaires. While they all trained and played at the highest level 
of performance for their age, there were, among others, differences in their 
interval endurance capacity as well as their tactical skills. The longitudinal 
study showed that at the age of 12, some players lacked game intelligence. 


According to their trainers, they had difficulty not only in “knowing what to 


do” on the field (e.g. declarative knowledge) but especially in “doing it” (e.g. 
performing the right action at the right moment; procedural knowledge) 
(Elferink-Gemser, Kannekens, Lyons, Tromp, & Visscher, 2010). However, 
they scored very well in their interval endurance capacity (Elferink-Gemser et 
al., 2006). In youth competition at that age, the game is not yet played very 
fast in comparison to higher levels of performance. This means that in case of 
a wrong decision a player is still able to make up for it when he or she is 
motivated and has great interval endurance capacity. For example, in the heat 
of the moment a player runs to the right of the field, while it would be a 
better option to go through the middle to create a scoring opportunity. From 
teammates, trainers, coaches, and audience along the field, the player hears 
the instructions and changes direction to the middle of the field. In other 
words, the player was able to compensate for a lack of tactical skills by great 
interval endurance capacity and motivation. However, when the level of 
performance in youth competition moves up with age, it appears more 
difficult to compensate for game intelligence. Longitudinal group studies 
showed that with increasing level of performance, the group of athletes 
becomes more homogeneous in terms of their performance characteristics. 
All players need great motivation, interval endurance capacity, and tactical 
skills in order to survive the selection (Elferink-Gemser et al., 2006; 
Roescher, Elferink-Gemser, Huijgen, & Visscher, 2010; Kannekens, 
Elferink-Gemser, & Visscher, 2011). 

The multidimensional nature of performance development is also 
reflected in a retrospective study among an entire selection team of field 
hockey players winning a medal at the Olympic Games. Six months before 
the Olympic Games, they were asked to divide 100 percent over several 


categories of personal performance characteristics related to what they 


believed was necessary to reach the top in field hockey. The 
multidimensional performance characteristics which the players could choose 
from are based on a limited number of determining factors for elite field 
hockey, i.e. anthropometric, physiological, technical, tactical, and 
psychological (Elferink-Gemser et al., 2004). The results showed that the 
Olympic medal winners believed that a combination of several performance 
characteristics was needed to become an expert and also to stay at the highest 
level of performance. Anthropometric characteristics were considered least 
important (less than 10 percent), followed by physiological, technical, 
tactical, and, especially, psychological skills (more than 30 percent). 
Psychological skills were said to be even more important to maintain 
expertise in comparison to acquiring expertise. Although it is debatable how 
far these expert field hockey players were able to give an accurate estimation 
of the hierarchy of the performance characteristics necessary to acquire and 
maintain expertise, it does give an impression about the relevance of 
considering the multidimensional nature of expertise. When preparing a 
longitudinal group study, one may consider setting up a retrospective study 
design among experts beforehand to get insight into which performance 
characteristics are relevant to include in the longitudinal study. 

Along with the relevance of applying a multidimensional approach, 
monitoring athletes over time seems inevitable when one aims to understand 
underlying mechanisms related to acquiring and maintaining expertise. As 
shown in Figure 16.1, the development toward expert performance takes 
place during adolescence. Although the figure may suggest otherwise, the 
development of a sports career toward expert performance is not a linear 
process (Cobley, Till, O’Hare, Cooke, & Chapman, 2014; Phillips, Davids, 


Renshaw, & Portus, 2014). In the decade preceding expert performance, 


athletes improve their performance characteristics and sport performance not 
only by accumulating deliberate practice but also because they are maturing 
toward adulthood. An example is their physical growth spurt. Around the age 
of 12, on average girls experience a greater velocity regarding increase in 
height (i.e. peak height velocity, PHV). For boys, this is on average around 
the age of 14 (Carvalho et al., 2011). Because of the growth spurt, an athlete 
becomes physically better suited for the tasks at hand and as such can further 
improve his sport performance. This is especially the case in sports in which 
physical performance characteristics are highly related to performance, like in 
soccer or ice-hockey. 

There are also exceptions to “the rule.” For example in gymnastics it is 
known that a smaller stature is beneficial for performance (Ackland, Elliott, 


& Richards, 2003). Important to realize is that also regarding the growth 


spurt, large inter-individual differences are apparent (Malina, 2003). 
Although most adolescents are “average maturers” (i.e. experience their 
growth spurt within one year of the average age for PHV), some youth 
athletes are early maturers (i.e. experience the growth spurt at least one year 
earlier than average) whereas others are late maturers (i.e. experience the 
growth spurt at least one year later than average). The timing and tempo of 
maturation is related to an athlete’s performance development (Malina, 
2003). Taking all of this together, it seems wise to measure and monitor an 
athlete’s multidimensional performance characteristics together with his 
performance development over time. It seems that it is hard to capture the 
acquisition of expert performance by applying solely mono-disciplinary or 
cross-sectional research designs. Although these studies are of great 


importance, the value of combining knowledge from a variety of research 


domains in multidisciplinary studies in which youth athletes are monitored 


over time in a longitudinal research design seems promising. 


Studying General Performance Characteristics 


Improvement of performance is related not only to the number of training 
hours but to a large extent to the quality thereof. The deliberate practice 
theory of Ericsson and colleagues (Ericsson, Krampe, & Tesch-R6mer, 1993) 
is widely acknowledged in this respect and is applicable in a variety of fields 
of expertise. When studying the acquisition and maintenance of expertise, it 
is important to realize that improvement of performance toward excellence 
starts with a young individual having some ideas of the long-term goal to 
pursue (Ertmer & Newby 1996). This does not necessarily have to be a 
conscious goal; it can also be a rather vague idea of what to reach or “where 
to go” in the future (personal communication with Olympic champions 
Leontien Zijlaard-Van Moorsel in cycling and Marianne Timmer in speed 
skating). 

This long-term goal is broken up into numerous short-term goals along 
the road to success. Research has shown that there are several ingredients to 
being successful in reaching the short-term goals which can be broken down 
into “to know, to want, and to do.” The starting point is to have a clear 
understanding of what the requirements of the goal to attain are. What is 
needed to perform at a high level of performance in my specific sport? For 
this, an athlete needs to reflect on his own current performance and 
performance characteristics. What am I already good at? What are points for 
improvement? Reflection is a metacognitive skill, a higher order cognitive 


function (Huijgen et al., 2015), and relates to how an athlete sets goals to 


improve, based on _ experiences, strengths, and weaknesses. Other 


metacognitive skills which are relevant in this respect are planning (“to 


determine a problem before beginning to solve it”), monitoring (“keeping 
track of the progress during execution”), and evaluation (“checking the 
learning process and the result achieved after execution”) (Jonker, Elferink- 
Gemser, Tromp, Baker, & Visscher, 2015). Reflection, planning, monitoring, 
and evaluating the process only work if an athlete is motivated to improve 
and takes action to improve. He or she has to put effort into the continuous 
process of reaching expertise and believe that he or she can actually reach the 
goal (self-efficacy). The entire process is called self-regulation of learning 
which is defined as “the degree to which learners are metacognitively, 
motivationally, and behaviorally active participants in their own learning 
process” (Zimmerman, 1986, p. 308, 1989, p. 329). Self-regulation of 
learning has been associated with success in all kinds of sports (Toering, 
Elferink-Gemser, Jordet, & Visscher, 2009; Jonker, Elferink-Gemser, 
Toering, Lyons, & Visscher, 2010; Elferink-Gemser et al., 2015) as well as in 


other fields than sports, such as academics (Nota, Soresi, & Zimmerman, 
2004). Examples of methods to study self-regulation of learning are 
questionnaires (e.g. Self-Regulation of Learning Self-Report Scale (Toering, 
Elferink-Gemser, Jonker, Van Heuvelen, & Visscher, 2012)) or observations 
(Toering et al., 2011). 


Third Challenge: Choice of Appropriate Study 
Methods and Statistical Analyses 


Methodology 


The choice for the methodology and the statistical analyses of a study greatly 
depends on the nature of the research question. For example, if the goal is to 
investigate the relationship between technical skills and sports performance 
in youth soccer players, one of the first things that has to be determined is the 
type of study that will give most information about that relationship. Is the 
goal to compare technical skills among different populations of elite youth 
soccer players from a talent development program and sub-elite youth soccer 
players from a club team at the same point in time? Or is the goal to measure 
technical skills in a single population of elite youth soccer players over an 
extended period of time? The first approach is a typical cross-sectional study. 
The second requires a longitudinal study. In the example of youth soccer 
players, technical skills would be measured in elite and sub-elite players 
along with other characteristics that might be of interest such as age and 
gender. The defining feature of a cross-sectional study is that it can compare 
different population groups, i.e. elite and sub-elite, at a single point in time, 
like taking a snapshot. In line with research in this domain (e.g. Huijgen, 
Elferink-Gemser, Lemmink, & Visscher, 2014) elite youth soccer players 
outscore their sub-elite counterparts on technical skills tests. However, 
because cross-sectional studies do not consider what happens before or after 
the single measurement, it is uncertain if the elite youth soccer players 
improved their technical skills because of their training in the talent 
development team or whether they had been selected for the talent 
development program because of their superior technical skills. 

To better understand possible differences, a longitudinal study design 


may be a better choice. In the example, not just the soccer performance 


would have to be measured repeatedly, but also the technical skills (e.g. 
Huijgen, Elferink-Gemser, Ali, & Visscher, 2013). When setting up a 
longitudinal study, a researcher also has to think about the frequency of 
measuring. The number of measurements can vary between the performance 
characteristics. Some characteristics may be relatively stable (e.g. self- 
regulation of learning), whereas other performance characteristics are subject 
to change to a much larger degree (e.g. sprinting performance), whether this 
concerns an improvement or decrement. As a consequence, sometimes 
measuring longitudinally implies measuring quite often (e.g. every month), 
whereas in other cases measuring longitudinally can mean measuring “only” 


once a year or even with a longer time interval between measurements. 


From Mono-Disciplinary to Multidisciplinary and from Cross- 
Sectional to Longitudinal Study Designs 
From a historical point of view in sports science, the transition from a 
traditional model of talent identification and talent development (i.e. mono- 
disciplinary and cross-sectional) to a more sophisticated model (i.e. 
multidimensional and longitudinal) coincides with the type and number of 
scientific studies published in this field. To illustrate this, a concise literature 
search was performed in the databases Web of Sciences and PubMed. 
Articles related to talent identification and talent development in sports with a 
multidimensional approach, longitudinal approach, or a combination of both 
approaches were searched per time frame of five years from 1990 till 2015. 


The following keywords were entered in the databases: 
Multidimensional approach: 


(acquisition* OR maintenance* OR expert* OR talent*) AND (sport 
OR athlet* AND (multidisciplinary* OR multidimensional*)) 


Longitudinal approach: 


(acquisition* OR maintenance* OR expert* OR talent*) AND (sport 
OR athlet*) AND (longitudinal*) 


Combination of both approaches: 


(acquisition* OR maintenance* OR expert* OR talent*) AND (sport 
OR athlet*) AND (multidisciplinary* OR multidimensional*) AND 
(longitudinal*) 


Articles were included when an empirical study was performed and when 
participants were high-level athletes. Figure 16.2 shows an overview of the 
included articles per approach, with an increase in multidimensional and 


longitudinal research over time. 
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Figure 16.2 Overview of articles applying a multidimensional approach, 
longitudinal approach, or a combination of both approaches in the last 25 


years. 


From the overview above it becomes clear that over time an increasing 
number of studies apply a multidisciplinary and longitudinal study design 
when investigating the acquisition of expertise. When designing a scientific 
study, not just the methodology is important to consider but also the 


Statistical analyses to answer the research question. 


Statistical Analyses in Longitudinal Studies 


A variety of statistical analyses have been applied in longitudinal studies. 
From the 34 longitudinal studies published in PubMed and/or Web of 
Sciences in the last five years, repeated (multivariate) analyses of 
(co)variance was applied 14 times (Bush, Barnes, Archer, Hogg, & Bradley, 
2015; Crotin, Forsythe, Karakolis, & Bhan, 2014; Deprez et al., 2015a; 
Deprez, Fransen, Lenoir, Philippaerts, & Vaeyens, 2015b; Diaz et al., 2012; 
Feichtinger & Honer, 2015; Gonaus & Miiller, 2012; Gillich & Emrich, 
2014; Matthys et al., 2013; Till, Cobley, O’ Hara, Chapman, & Cooke, 201 3a; 
Till, Cobley, O’Hara, Cooke, & Chapman, 2014; Till, Jones, & Geeson- 
Brown, 2016; Van der Sluis et al., 2014; Williams, Oliver, & Faulkner, 
2011), followed by the calculation of effect sizes (11 times) (Bush et al., 
2015; Deprez et al., 2015a; Deprez et al., 2015b; Feichtinger & H6ner, 2015; 
Giillich & Emrich, 2014; Hancock, Young, & Ste-Marie, 2011; Saward, 
Morris, Nevill, & Sunderland, 2015; Till et al., 2013a, 2014, 2016; Van der 
Sluis et al., 2014), multilevel modeling (9 times) (Bidaurrazagna-Letona et 
al., 2015; Deprez et al., 2014; Deprez et al., 2015c; Hellard et al., 2013; 
Huijgen et al., 2013; Saward et al., 2015; te Wierike et al., 2014; Valente- 
dos-Santos et al., 2014; Van der Sluis et al., 2014), correlations (7 times) 


(Bidaurrazagna-Letona et al., 2015; Crotin et al., 2014; Deprez et al., 2015a; 
Moreira, Crozara, Goethel, de Paula, & Vieira, 2014; te Wierike et al., 2014; 
Vink, Raudsepp, & Kais, 2015; Williams et al., 2011), and dependent 
samples tests (6 times) (Deprez et al., 2015b; Diaz et al., 2012; Feichtinger & 
Honer, 2015; Huijgen et al., 2013; Till et al., 2016; Van der Sluis, Elferink- 


Gemser, Brink, & Visscher, 2015). Regression analyses have been reported 4 


times (Crotin et al., 2014; Deprez et al., 2015b; Giillich & Emrich, 2014; 
Nybo, Schmidt, Fritzdorf, & Nordsborg, 2014) and analyses with Z-scores 


have been reported 3 times (Cobley et al., 2014; Moreira et al., 2014; Till, 


Cobley, O’Hara, Chapman, & Cooke, 2013b), while discriminant analysis 
(Gonaus & Miiller, 2012; Matthys et al., 2013), structural equation modeling 


(Ivarsson et al., 2015; Zuber & Conzelmann, 2014), and latent growth 
modeling (Ivarsson et al., 2015; Vink et al., 2015) have been applied 2 times 
each. Finally, there were statistical analyses which have been reported in one 
study so far, being principal component analysis (Gonaus & Miiller, 2012), 


interview analysis (Tibbert, Andersen, & Morris, 2015), factor analysis 


(Feichtinger & Honer, 2015), chi-square goodnesss-of-fit tests (Hancock et 
al., 2011), and rates of improvements (ROI) (Deprez et al., 2015a). 

As such, the top two statistical analyses that researchers recently applied 
in longitudinal studies on acquiring expertise are (1) repeated (multivariate) 
analyses of (co-)variance and (2) multilevel modeling. In addition, 
calculation of effect sizes often serves as a support for these and other 


Statistical analyses. 


Repeated (Multivariate) Analyses of (Co-)variance 


In the repeated analyses of variance, whether they are multivariate, with or 
without covariate, Means are compared to each other (Field, 2005). In such 
analyses, it can be investigated whether several independent variables interact 
with each other (e.g. the moment of testing, different levels of performance, 
sex, age) and what effects these interactions have on the dependent variable 
(e.g. sport performance or one or more performance characteristics). The 
pitfall is that a complete dataset is required to do the analyses. In case of 
missing data on one or more of the variables, a participant’s scores are 
excluded from the analyses. In addition, the moment of testing has to be 
equal for all participants, which in practice is often not the case. The 
calculation of effect sizes is an addition to a statistical analysis in which 
significance is tested, such as a repeated analysis of variance. The reason to 
have an additional test is that when a test statistic is significant, this does not 
necessarily mean that the effect it measures is meaningful or important (Field, 
2005). An effect size is an objective and standardized measure of the 
magnitude of the observed effect. Because the measure is standardized, it is 
possible to compare effect sizes across different studies that have measured 
different variables, or have used different scales of measurement (Field, 
2005). In longitudinal studies, they can be used to interpret any difference in 
score between two or more measurements within the same participant (e.g. 
sprint time at the start of the competitive soccer season compared to sprint 


time near the end of that season). 


Multilevel Modeling 


Multilevel modeling solves some of the problems stated above. It makes it 
possible to address research questions and data structures which involve more 
than one type of unit. These analyses originated in studies involving several 
levels of aggregation, such as individuals in countries, or pupils in classrooms 
in schools (Snijders & Bosker, 2000). Historically, Robinson (1950) started 


the discussion of the ecological fallacy, where associations at one level of 


aggregation were mistakenly regarded as evidence for associations at a 


different aggregation level (see Alker, 1969 for an extensive review). This led 


to interest in how to analyze data including several aggregation levels. 
Repeated measurements within an individual can also be regarded as two 
different aggregation levels, i.e., a nested structure, with the measurements at 
the lowest level of analysis and the athletes at the hierarchical higher level. A 
suitable model for longitudinal data is the hierarchical linear model which is a 
type of regression analysis for multilevel data where the dependent variable is 
at the lowest level. Explanatory variables can be defined at any level 


(Snijders & Bosker, 2000). As such, multilevel modeling is an extension of 


multiple regression, which is appropriate for analyzing hierarchically 
structured data. In longitudinal datasets, a simple two-level hierarchy can be 
defined with the repeated measurements (defined as level 1 units) grouped 
within the individual youth athletes who form the level 2 units. 

An advantage of using a multilevel regression modeling approach is that 
both the number of measurements and the temporal spacing of the 
measurements may vary between players (Maas & Snijders, 2003). A 


multilevel model not only describes underlying population trends in a 


response (the fixed part of the model), but also models the variation around 
this mean response due to the time of measurement and due to individual 
differences (the random part) (Snijders & Bosker, 2000). An example is a 
longitudinal study on the development of the interval endurance capacity in 
217 elite and sub-elite youth field hockey players aged 12-19 (Elferink- 
Gemser et al., 2006). Following Snijders and Bosker (2000), the first step in 
the multilevel modeling of the interval endurance capacity was to establish a 
satisfactory variance structure for these longitudinal data, using age 
(measured as months/12—15 years). Then, the difference between elite and 
sub-elite groups was modeled, taking into account interactions with age and 
gender. In a next step, the effects of the anthropometric variables, height (m), 
lean body mass (kg), and percentage body fat, were investigated. 
Subsequently, the effect of the total number of training hours per week, as 
well as the effect of different types of training (distinguishing field hockey 
training and additional training) were investigated. Finally, the effect of 
motivation was tested. With the model, scores on the Interval Shuttle Run 
Test could be predicted for elite and sub-elite boys and girls in field hockey 
in the age-band of 12—19 years. The multilevel analyses made it possible to 
reveal that during adolescence both male and female elite youth players have 
a more promising development pattern of their interval endurance capacity 
than sub-elite youth players (Elferink-Gemser et al., 2006). The longitudinal 
changes in interval capacity were investigated using the multilevel modeling 
program MIwiN (Goldstein et al., 1998) but recent versions of SPSS also 
provide the possibility to create a multilevel model. 

Although in multilevel modeling it is possible to include participants 
with a different number of measurements as well as a different time interval 


between measurements, there are still challenges to face when designing 


longitudinal studies. One of them applies to participant follow-up, as 
mentioned earlier in this chapter. There is a risk of bias in case of dropout of 
study participants, for example because they have been de-selected from a 
talent development program. If participants who are followed to the planned 
end of the study differ from participants who discontinue follow-up, the 
results may not be representative of the original target population. Another 
challenge relates to time-varying covariates. Although longitudinal designs 
offer the opportunity to associate changes in exposure to, for example, 
training on technical and tactical skills in a talent development program with 
changes in the outcome of interest (e.g. sports performance), the direction of 
causality can be complicated by “feedback” between the outcome and the 
exposure. For example, in an observational study of the relation between 
technical and tactical skills on soccer performance, a talented player’s current 
level of performance may influence the quality and quantity of training in the 
future, and as such a player’s level of performance. Without an experimental 


design it remains difficult to disentangle causal relationships. 


Multidisciplinary Longitudinal Studies 


Although not often, a few studies focusing on the acquisition of expert 
performance combined a multidisciplinary and longitudinal study design (see 
Table 16.1), One of the great challenges of such designs is the attrition of 
participants. Especially when it comes to youth athletes who may or may not 
be selected for the youth selection team each subsequent year, it is difficult to 
monitor their progress and measure them repeatedly if they leave the team. 
The studies in Table 16.1 have a duration between three and seven years and 
organized three to fifteen measurement occasions. The number of participants 


varied between 43 and over 500. 


Table 16.1 Overview of studies on the acquisition of expertise with a 


multidimensional and longitudinal approach 


Participants 
(age in Multidimensional Longitudinal Number 
years; n) design design measuremi 
Spamer South Anthropometrical, Five-year 6 
& Hare Africanmale physical and period 
(2001) rugby motor skills 
players (10- 
year-old; n = 
43; 16-year- 


old; n = 29) 


Elferink- 
Gemser 
et al. 
(2006) 


Dutch male 
and female 
youth field 
hockey 
players 


(12-19 
years; n = 
217) 


Anthropometrical, Three-year 
physiological and __ period 
psychological 

characteristics 


Elferink- 
Gemser 
et al. 
(2007) 


Matthys 
et al. 
(2013) 


Dutch male 
and female 
youth field 
hockey 
players 
(14.20 + 1.1; 
n= 65) 


Belgian 
male youth 
handball 
players 
(oldest 
group: 
15-17, n= 
41; youngest 
group: 
13-15, n= 
53) 


Anthropometrical, 
physiological, 
technical, tactical, 
and psychological 
characteristics 


Three-year 
period 


Anthropometrical, 
maturity and 
physical 
characteristics 


Three-year 
period 


Deprez 
et al. 
(2014) 


Valente- 
dos- 
Santos et 
al. 


Flemish 
male youth 
soccer 
players 


(12.20 + 1.3; 
n= 162) 


Portuguese 
male youth 
soccer 
players 


Anthropometrical, 
maturity, motor 
coordination, and 
soccer-specific 
aerobic 
performance 


Anthropometrical, 
maturity, physical, 
and technical 
characteristics 


Five-year 
period 


Five-year 
period 


14 


(2014) 


Deprez 
et al. 
(2015b) 


(11-13; n= 
83) 


Belgian 
male youth 
soccer 
players 


(8-16; n= 
388) 


Anthropometry Six-year 
and maturity, period 
physical fitness, 

and motor 

coordination 


Deprez 
et al. 
(2015c) 


Belgian 
male soccer 
players 


(7-20; n= 
595) 


Anthropometry, Seven-year 
motor period 
coordination and 

jumping 

performance 
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The studies in Table 16.1 all applied a multidimensional and 
longitudinal study design. The value of multidimensional, longitudinal 
research is not to be underestimated. It provides insight into the hierarchy of 
multidimensional performance characteristics important for level of 
performance and can answer the question of which performance 
characteristics are important at which age. A key strength of longitudinal 
studies is the ability to measure change in outcomes and/or exposure at the 
individual level. It provides the opportunity to observe individual patterns of 
change. Such studies can also reveal that there are several pathways to the 
top. 

More than mono-disciplinary, cross-sectional studies, these kinds of 
study designs focus on the complicated development of youth athletes toward 
expert levels of performance. When we go back to the earlier mentioned 
example of the studies on high-performing youth field hockey players 
(Elferink-Gemser et al., 2006, 2007), a multidimensional, longitudinal design 


can give a different perspective on level of sports performance than a mono- 
disciplinary cross-sectional design. With all players being selected for a 
prestigious talent development program of a professional field hockey club in 
the Netherlands, elite female players scored better than sub-elite female 
players on a test for interval endurance capacity when they were 16. 


However, this was not the case when they were 12. At that younger age, the 


sub-elite players either scored similar to or even better than the elite players. 
In the subsequent years, the elite players were able to catch up in terms of 
their endurance and improved more each year. This improvement was shown 
to be related to their percentage body fat, motivation, and additional training 
hours. As such, the interval shuttle run test may be a good test to monitor the 
development of youth field hockey players but may not be a good choice for 
talent selection at the age of 12. It is hypothesized that the scouts deciding on 
who is to join the selection teams paid attention to, among other 
characteristics, the interval endurance capacity of the players. Sub-elite 
players may have lacked tactical and technical skills but were able to 
persevere in the games, catching the eye of the scouts who were observing 
players alongside the fields. This also touches upon the difference in how 
performance outcome is measured between various sports. Multidimensional, 
longitudinal studies can provide insight in relevant performance 
characteristics to the people working with athletes aiming to reach and 


maintain excellence. 


Concluding Remarks 


Although this chapter focused mainly on the field of sports, a similar 
approach can be taken in other fields of interest. Multidimensional, 
longitudinal studies can provide valuable insight in measures of performance 
and relevant performance characteristics. Even more, such studies can shed 
some light on the development thereof. This insight can be used by the 
individuals themselves to further improve their performance and by people 
working with young individuals aiming to reach and maintain excellence. 
This does not mean, however, that mono-disciplinary, cross-sectional studies 
are not valuable in this respect. To the contrary, it is especially those studies 
that are able to investigate more in-depth and up-to-date the characteristics 
that actually make a difference between different levels of performance 
toward expertise, which makes them indispensable. The choice for which 
variables to include in a longitudinal study is difficult and based on the state 
of the science at that moment. To be able to make intra-individual 
comparisons during youth, changing the measurements along the way in a 
longitudinal study is not advocated. As such, a longitudinal study design is 
not as flexible as cross-sectional studies when it comes to including the latest 
insights. 

To end, by combining several study designs, the work toward 
disentangling the mysteries of acquiring and maintaining expertise is 
progressing. The next steps to take are individual profiling of the experts’ 


development of performance as well as the underlying multidimensional 


performance characteristics taking into account individual differences in 


reaching the top. 
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Introduction 


Understanding how experts perform, and perform exceptionally, is not only 
of theoretical interest, it is a program of research with practical implications 
for the “real world.” Studies of expertise inform the design of educational and 
occupational training programs (e.g. Lajoie, 2009). Studies of expertise, 
moreover, inform job design (e.g. Xiao, Milgram, & Doyle, 1997), 
development of decision support systems (e.g. Dérner & Schaub, 1994), and 
error Management procedures (e.g. Robledo et al., 2012). Indeed, one can 
argue few other topics are of the same practical importance as attaining an 
understanding of expert performance (Gladwell, 2008). 

The substantive, and practical, implications of our understanding of 
expert performance has led to the development of a number of methods that 
might be used to identify what experts do, and when they do it, to bring about 
their exceptional performance. One might compare the verbal statements 
made by “experts” and novices as they work on a task (e.g. Ericsson & 
Simon, 1984). One might seek to identify critical contingencies shaping 
expert performance (e.g. Ericsson & Smith, 1991). One might seek to identify 
the unique errors made by experts in contrast to novices (e.g. Hershey, 
Walsh, Read, & Chulef, 1990). One might conduct experimental studies 
intended to induce changes in the rate at which expertise is acquired (e.g. 
Charness, Tuffiash, Krampe, Reingold, & Vasyukova, 2005). 

We do not wish to dispute the merits of these, and other, techniques that 
might be used to understand the nature of expert performance. By the same 


token, experts are unique. More specifically, by virtue of their exceptional 


work in a given domain, and social recognition of the value of this work, 
experts leave behind them a record, often objective records, of their work. In 
some cases, these records are embedded in documents developed in the 
course of a project being pursued by an expert — for example, design 


documents (Weisberg, 2011). In other cases, these records are summarized in 


historic reviews examining how an expert developed a noteworthy product, 
such as the airplane or the telephone (Carlson & Gorman, 1992; Crouch, 
1992). In still other cases, observations of expert performance can provide 
records describing what was accomplished and how it was accomplished (e.g. 
Dunbar, 1995). 

The wide variety of material available describing their work is 
noteworthy because it points to another method that might be used to 
understand the nature of expertise. Intensive study of the records bearing on 
the performance of a single expert might be used to gain some understanding 
of expert performance. Indeed, such analyses, case analyses, may focus on 
one expert, multiple experts, one event, or multiple events. And, because 
cases provide rich, “real-world” descriptions of expert performance, cases 
may provide a uniquely valuable basis for understanding expert performance 
(Hunt, 2009; Mumford, Friedrich, Caughron, & Antes, 2009). 

Although cases apparently provide an attractive vehicle for the study of 
expert performance, application of case data has been questioned. For 
example, although cases are rich sources of data, there are only a limited 
number of cases being examined (Parry, Mumford, Bower, & Watts, 2014). 
Cases represent incidents of exceptional performance but not necessarily 
normal performance (Mumford, 2006). Case records are selective and may 
not be fully accurate (Bedell-Avers, Hunter, Angie, Eubanks, & Mumford, 


2009). Embedded in cases are a complex set of events where it may prove 


difficult to isolate critical causes even when comparison groups, case 
comparisons, are available (Ligon, Hunter, & Mumford, 2008). 

Given questions about generality, accuracy, typicality, and causal 
isolation, it seems that the traditional reasons we do not employ cases in the 
sciences also stand in the study of expert performance. By the same token, 
the real-world nature of cases and the richness of the data embedded in cases 
suggest that they provide data of unique ecological validity. As a result, cases 
might provide an appropriate starting point for studies of expertise. And, 
when case observations are confirmed, through triangulation using other 
methods, they might provide a strong base for the study of expert 
performance. In the present effort, we will examine how cases, and case 
triangulation, have been used to elucidate certain critical features of expert 
performance. Before turning to the specific examples, it would seem germane 


to briefly review what we mean by the terms cases and case method. 


Case Method 


Defining Cases 


Within the field of psychology, the traditional view of cases, and case studies, 
originated in the clinical area. For clinicians, and psychologists in general, a 
case was viewed as observations of a single individual, typically by a single 
clinician, based on behavior, or events, as reported by this individual. These 
observations, moreover, were typically unstructured with respect to both the 
target individual and the observer. The idea here was that open, unstructured, 
observation would allow the origins of the issue, problem, or difficulty to 
emerge. Although, as psychologists, we adopt this view whenever we hear 
the word “cases,” the stereotype does not reflect the meaning of the word 
cases. 

A case reflects an event, or series of events, occurring in a “real-world” 
context (Mumford, 2002). Events reflect an action, or set of actions, taken 
with respect to environmental contingencies (Pillemer, 1998). As a result, 
cases may be, but are not necessarily, tied to a particular individual. Thus, 
one might study firms with respect to cases of innovation (Dougherty & 
Hardy, 1996). Cases, moreover, do not presume intentionality on the part of 
the entity involved in acting with respect to these events (Weick, 1998). 
Furthermore, as the reactions of entities to events, cases are not unique. 
Instead, reactions may replicate themselves with respect to event types across 
people or context (Mumford, 2006). Finally, it should be recognized that the 
foregoing definition makes no assumptions about who makes observations or 
the structure of these observations (Parry et al., 2014). Thus observations 
may be structured or unstructured. 

It is of course not sufficient to say what a case is not. One must also 


define cases in a positive way — saying what a case actually is. As action, or 


actions in response to actual, or perceived, environmental contingencies, 
cases inherently refer to “real-world” events. Indeed, the “real-world” nature 
of cases is one attribute of cases which makes them an attractive vehicle for 
studies of expertise and expert performance. Along related lines, cases, as a 
set of actions taken in response to “real-world” events, are inherently 
contextualized. Thus, in any case study, the specific nature of the events, 
and/or the event context of concern, must be specified. Moreover, how an 
entity acts with respect to these events must be specified. In studies of 
expertise, often, albeit not always, these entities will be people displaying 
substantial expertise. Finally, as noted above, cases involve observations 
about actions taken with respect to “real-world” events. As is the case with 
any other observational method, agreement among judges must be 
demonstrated on attributes (e.g. frequency, intensity) of concern using the 
source material providing descriptions of these actions in context. 

These observations about the nature of cases are noteworthy because 
they point to the critical methodological issues to be considered in any case 
study. Perhaps the first, and most important, issue to be considered in the 
nature of the events to be examined is the case study (Yammarino, Mumford, 


Serban, & Shirreffs, 2013). For example, events may be drawn from incidents 


of chess master performance (de Groot, 1978) or from incidents of 
experienced pilot performance (Schreiber et al., 2009). Not only must the 
domain from which events will be drawn be specified, the nature of 
legitimate, and illegitimate, events must be specified due to the variety of 
events occurring in any domain. Thus, in studies of leadership using the case 
method, legitimate events are drawn from the person, the leader, when the 


leader was at the pinnacle of power (Mumford, 2006). Events bearing on 


early development, while relevant to leader development (Ligon et al., 2008), 


do not bear on leadership or the exercise of influence (Yukl, 2011). 


Source Selection 


Following definition of legitimate and illegitimate events, case studies 
depend on identification and appraisal of actions taken in response to these 
events. Although case studies have often been based on a professional’s 
analysis of actions reported by an individual in response to an event, a variety 
of other sources of information exist that allow definition of events and 
analysis of actions taken with respect to these events. For example, one might 
employ biographies, validated academic biographies of eminent individuals, 


in studies of expert performance (Barrett, Vessey, Griffith, Mracek, & 


Mumford, 2014). Alternatively, for other purposes, for example experts’ 


motivations, other sources, sometimes autobiographies (Zeni, 2014), might 


be employed. However, a wide variety of sources, beyond biographies and 
autobiographies, are available that have been employed in case studies, 
including work design documents (Weisberg, 2011), work products (Ramey 
& Weisberg, 2004), oral history (Rich & Weisberg, 2004), obituaries 
(Mumford et al., 2005), Wikipedia entries (Yammarino et al., 2013), product 


plans (Xiao et al., 1997), and direct observation (Dunbar, 1995). 


With regard to the source material used in case studies, it is important to 
bear in mind three points. First, certain sources provide information bearing 
on particular events and actions. Thus, evidence should be provided 
indicating that source material does, in fact, reflect the events and actions of 


interest (Parry et al., 2014). Second, sources, based on their internal 


structures differ in where and how actions and events are described. Thus a 
substantive and structural justification should be provided for the particular 


“locations” from which source material was drawn (Mumford, 2006). Third, 


sources differ in the accuracy with which they report certain actions and 
certain events. Thus evidence should be provided for the accuracy of sources 


in reporting actions or events (Bedell-Avers et al., 2009). 


Abstraction Procedures 


Following source selection, one must identify the specific procedures for 
abstracting events from source material and the actions embedded in the 
events. Ideally, systematic procedures for identifying relevant events should 
be specified, and evidence should be provided for inter-judge agreement in 
executing these procedures. An illustration of the development of systematic 
procedures for abstracting collective leadership events from biographies of 
George Marshall may be found in Friedrich et al. (2014). Systematic 
identification of events is of special importance when case studies are being 
conducted because the ultimate unit of analysis is the event, not the 
individual, and so we must know how events were specified. Following event 
specification, actions embedded in these events are abstracted. Actions 
embedded within events provide the key observational data employed in case 
studies. As a result, the procedures used to abstract actions within events 
should be specified. And evidence should be provided for the reliability with 
which those procedures were executed and the validity of these procedures 


for abstracting actions from events (Parry et al., 2014). 


Interpretation of Cases 


With identification of the events to be analyzed, and abstraction of the actions 
taken by an entity in response to these events, the base material needed for a 
case study is available. What must be recognized here, however, is that 
judges’ appraisal of actions is inherently contextually based. Thus in 
appraising cases, actions are appraised within the context of events. Often 
those appraisals are made by a single judge, although many studies use 


multiple judges (Mumford, 2006). Regardless of whether a single judge or 


multiple judges are employed, judges’ background qualifications, and 
expertise, should be established because judges’ mental models will influence 
appraisals of events and actions taken with respect to these events (Rouse & 
Morris, 1986). 

Often the interpretation of case studies, either single or multiple event 
studies, is based on a single judge’s qualitative interpretation of observations 


(Gergen, Josselson, & Freeman, 2015). In other case studies, judges, typically 


multiple judges, are asked to appraise the actions evident in case events with 
respect to the frequency or intensity with which certain attributes, or 
dimensions, are evident in actions manifested in a case. Thus O’Connor, 
Mumford, Clifton, Gessner, and Connelly (1995) appraised the actions taken 
by leaders with respect to narcissism and power motives. Mumford (2006) 
asked judges to appraise cases of leader problem-solving with respect to key 
processes, such as problem definition and information gathering. Vessey, 
Barrett, Mumford, Johnson, and Litwiller (2014) appraised the need for 
planning skills (e.g. forecasting, backup plans) among those asked to lead 


creative ventures in the sciences. 


When multiple judges are used to appraise actions in case events, 
evidence should be provided for inter-judge agreement and reliability. 
Additionally, of course, evidence should be provided pointing to the validity 
of judges’ appraisals — often through convergent and divergent validity 


evidence (Mumford et al., 2007). Typically reliable and valid appraisals of 


attributes of actions in case events are obtained when judges have been given 
adequate training. Viable training programs are relatively long — 20 hours to 
40 hours. In these training programs, judges are (1) familiarized with the 
nature of the source material, (2) familiarized with relevant events and the 
nature of the actions embedded in these events, (3) presented with the rating 
scales to be used in appraising case actions, (4) provided with examples 
bearing on application of these rating scales in appraising actions with respect 
to case events, (5) asked to practice applying these rating scales to a set of 
sample products, and (6) in practice they are asked to assess and resolve any 
disagreements. Although application of these procedures has typically 
resulted in appraisals of case attributes evidencing adequate reliability and 
validity (Parry et al., 2014), different cases, and case events, may require 


different training procedures. 


Criteria 


Many case studies are essentially descriptive in nature. Thus in studies of 
expertise, one might describe the performance of experts, or alternatively, 
contrast descriptions of expert performance with the performance of novices 
(Isenberg, 1986). And, in fact, if appropriate comparison groups have been 
defined, such descriptive studies might be highly informative. Case events, 
however, typically have outcomes. Assessment of these outcomes for a 
particular case, or across a series of case events, allows a stronger set of 
inferences to be drawn from case studies. Thus, Vessey et al. (2014) assessed 
the performance of scientific leaders, a series of case analyses, with respect to 
h-indices, the number of creative products produced, and number of awards 
received. In another study, Yammarino et al. (2013) assessed leader cases, 
noteworthy political leaders, with respect to assassination attempts and 
successful assassinations. In still another study, Eubanks et al. (2010) 
examined maintenance of reputation following cases of errors — mistakes 
made by historically notable leaders. 

Clearly criteria, or outcomes, can be examined in case studies if a 
sufficient number of cases are analyzed. And criteria employed in these 
studies can be especially useful given the ability to allow for time lags in 
assessing case outcomes. The studies cited above, however, point to three key 
conclusions with regard to the criteria employed in inferential (outcome 
based) case studies. First, in these studies, multiple criteria are employed. 
Second, the criteria employed are a mix of objective (e.g. h-indices) and 
subjective criteria (e.g. impact of leader on team). Third, if valid criteria are 


employed along with viable comparison groups, to avoid range restriction, 


analysis of cases may lead to reasonably strong inferences, inferences 
confirmed by studies using other methods (e.g. experimental, psychometric), 


with respect to the causes of performance (Mumford & Mecca, 2013). 


Controls 


The strength of the inferences drawn in this regard, however, is not solely 
dependent on criteria or outcome measures. The strength of the inferences 
that may be drawn from the systematic analysis of cases also depends on the 
control variables employed. Controls may be employed to remove extraneous 
variance in a criterion. In addition to this classic conception of control, at 
least two other types of control should be considered when cases are 
employed. One set of controls pertains to the nature of the events being 
examined — for example, time since observation, qualifications of the 
observer, and amount of detail provided. The other set of controls should 
consider socio-contextual variables influencing case reporting — for example, 
observers’ affective reactions to the case, expert evaluations of case accuracy, 
direct versus indirect observation. Indeed, inclusion of these controls not only 
permits stronger inferences, it allows extraneous explanations to be ruled out. 


Thus Mumford (2006), in a biographical study, found that the author’s liking 


for the principal involved in the case had little impact on the nature of case 
material provided or the relationship between rated case attributes and case 


outcomes. 


Expert Performance 


Having described cases, and the case method, a new question comes to the 
fore. Can the case method be employed in drawing inferences about expert 
performance? The answer to this question has been provided by both case 
studies and experimental studies of social innovation. Moreover, implications 
drawn from case studies and conclusions made from experimental studies 
triangulate our understanding of expert performance and help generate more 


reliable and accurate suppositions bearing on expert performance. 


Social Innovation Case Studies 


One illustration that case methods can be used to draw inferences about 
expert performance may be found in studies of social innovation (Jiang & 


Thagard, 2014; Mumford, 2002). The term social innovation refers to the 


formulation of original, high-quality, elegant solutions to complex, novel, ill- 
defined problems arising in a social context. Social innovations are unique 
due to their widespread impact on society as a whole. Thus, time motion 
studies (Kanigel, 1997), the iPhone (Isaacson, 2014), and Habitat for 


Humanity (Haun, Laurence, & Snyder, 2002) all represent noteworthy social 


innovations. And the impact of social innovations on our world poses a new 
question. Is there reason to suspect that expert performance might underlie 
social innovations? 

To address this question, Simonton (2015) examined the 1,093 patents 
obtained by Thomas Edison, a highly accomplished inventor, over the course 
of his career. To assess the patent trends throughout Edison’s career, 
Simonton divided the patents into eight categories of creative production and 
his career into five-year increments. It was found that Edison’s production 
follows a bimodal distribution, such that two distinct peaks of creative 
achievement are evident, contrary to most eminent inventors who 
demonstrate one early- to mid-career peak. In addition, one reason that 
Edison evidenced such remarkable creativity throughout his career, Simonton 
(2015) suggests, is based on his use of a “network of enterprises,” or his 
willingness to simultaneously work on multiple projects from interconnected 
subject areas. Thus, when a clear obstacle arose for one project, Edison could 


seamlessly shift his efforts to another project. Furthermore, on occasion, the 


ideas from one project could stimulate Edison’s production on an otherwise 
stagnant project. 

In another assessment of eminent social innovation, Mumford (2002) 
conducted a study of ten cases all drawn from Benjamin Franklin. Franklin 
was selected as the base for this study for three reasons. First, Franklin is 
historically noted for a number of social innovations made in colonial 
Philadelphia — e.g. founding subscription libraries, establishing a viable paper 
currency, establishing the University of Pennsylvania, developing franchising 
operations (Isaacson, 2004 ). Second, multiple cases of historically notable 
social innovations were available for one individual in a fixed context — 
colonial America prior to the French and Indian Wars. Third, Franklin, as a 
publisher, provided first-hand accounts of his thoughts as he developed each 
of these innovations with his thoughts regarding the majority of these 
innovations appearing in the Pennsylvania Gazette. 

Mumford (2002) conducted an unstructured study of each of these cases 
to identify the key factors shaping Franklin’s performance in solving social 
innovation problems. His analysis identified eight key characteristics evident 
in all of Franklin’s social innovations: (1) viable problem definition, (2) 
chaining innovations, (3) practicality, (4) financing, (5) persuasion, (6) 
rearrangement of social relationships, (7) analysis of key causes, and (8) 
forecasting the social consequences of action. With regard to expert 
performance, however, these observations point to three skills that might be 
critical in solving social innovation problems — problem definition, causal 
analysis, and forecasting. By the same token, this conclusion arises from 
study of a single person, albeit multiple cases involving this person, in a 
single historical period. To address the issue further, Mumford and Moertl 


(2003) conducted a qualitative study of two well-documented twentieth- 


century social innovations — development of standardized tests and scientific 
management. Relevant historic documents bearing on each of these cases 
were reviewed. It was found, using unstructured analytic techniques, that 
demonstration projects were critical components of each of those innovative 
efforts. More centrally, these demonstration projects were used to refine 
definition of key causes, and provide practical skills in manipulating these 
causes, along with allowing the “reality testing” meeded for viable 
forecasting. Thus expert performance in solving social innovation problems 


does appear to require both causal analysis and forecasting. 


Experimental Studies of Social Innovation 


The fact that forecasting and causal analysis skills appear critical to expert 
performance in solving social innovation problems, provided the basis for a 
series of experimental studies intended to test hypotheses flowing from these 
case studies. Indeed, these experimental studies were valuable because prior 
studies have highlighted the extent of experts’ skill in forecasting and their 
ability to isolate critical causes in solving “real-world” social innovation 


problems (Mintzberg, 1973; Pant & Starbuck, 1990). This attempt to confirm 


the conclusions emerging from these case studies began with initial efforts 
intended to demonstrate the impact of forecasting and causal analysis on 
solving social innovation problems. 

In an initial study along these lines, Dailey and Mumford (2006) 
presented undergraduates with three historic case studies describing social 
innovations in either education or public policy domains — all innovations 
unfamiliar to these undergraduates. Participants were asked to assume the 
role of a member of a proposal review panel and appraise each case with 
respect to resource requirements (e.g. personnel time, time to 
implementation) and outcomes (e.g. program acceptance, number of positive 
outcomes). Participants’ forecasts were contrasted with actual case outcomes 
and resource requirements drawn from historic records under conditions 
where implementation intentions, domain familiarity, and evaluation criteria 
were manipulated. It was found that forecasts of resource requirements and 
outcomes were accurate, indeed highly accurate, when participants had 
familiarity with the domain and believed their recommendations would be 


acted on by the foundation funding proposals. 


Not only has evidence been accrued for forecasting, in another study 
Marta, Leritz, and Mumford (2005) provided evidence for the impact of 
causal analysis on social innovation. In this study participants, again 
undergraduates, were presented with a business consulting problem requiring 
social innovation. Participants worked in teams in solving this problem, 
solutions were appraised for quality and originality, and participants 
nominated team leaders. Prior to starting work on this problem, however, all 
team members were asked to complete a measure of planning skills, which 
assessed identification of key causes. It was found that when team leaders 
evidenced skill in identifying key causes, better solutions to this social 
innovation problem were observed. 

Having provided initial demonstration of the relevance of forecasting 
and causal analysis to solving social innovation problems, additional studies 
were conducted to demonstrate the importance of these capacities. Studies by 
Hester et al. (2012) and Marcy and Mumford (2007, 2010) attempted to 
assess the impact of causal analysis on solving social innovation problems — 
with one study examining performance in solving an _ educational 
restructuring problem (Marcy & Mumford, 2007), one study examining 
performance in a university leadership simulation (Marcy & Mumford, 
2010), and one study examining performance in developing a marketing 
campaign (Hester et al., 2012). In all three studies, participants — 
undergraduates — were provided with training in viable causal analysis 
strategies (e.g. think about causes that have large effects, think about causes 
that have direct effects, think about causes that work together). In all three 
studies, it was found that training in causal analysis resulted in better 


solutions to social innovation problems vis-a-vis untrained controls. 


However, these effects were especially strong when accompanied by 
deliberation (Marcy & Mumford, 2007). 

In another set of studies, Byrne, Shipman, and Mumford (2010) and 
Shipman, Byrne, and Mumford (2010) focused on forecasting skills. In the 
Byrne, Shipman, and Mumford (2010) study, participants — undergraduates — 
were asked to work on a low-fidelity simulation task to generate solutions for 
marketing a new product, a high energy root beer, where marketing plans 
were evaluated for quality, originality, and elegance. In the Shipman, Byrne, 
and Mumford (2010) study, participants — again undergraduates — were asked 
to work on a low fidelity simulation task where visions, and plans for leading 
an experimental school were evaluated for quality, originality, elegance, 
perceived utility, and affective impact. As the participants reflected on these 
tasks they received emails from a consulting firm “hired” to help them 
develop their solutions. These emails requested forecasts of plan outcomes 
where forecasted outcomes were appraised by judges with respect to 21 
attributes, such as number of outcomes forecasted, forecasting negative 
outcomes, and forecasting long-term outcomes. These appraisals of 
participants’ forecasts were factored with four dimensions emerging, 
including forecasting resources, forecasting extensiveness, forecasting 
negative outcomes, and length of forecasting time frame. It was found that 
extensiveness of forecasts and forecasting time frame were strongly 
positively related to indices of success in solving these social innovation 
problems — producing positive relationships between 0.20 and 0.40, and 
stronger relationships than other individual difference variables such as 


intelligence and divergent thinking. 


Expert Social Performance 


Cases of Leader Performance 


The studies described above indicate that case analysis, based on either 
historic observations or analysis of work products, can be used to identify key 
variables, for example, causal analysis and forecasting, contributing to expert 
performance. Moreover, analysis of case studies apparently allows 
identification of certain key contextual variables, for example constraints, 
influencing expert performance. A variety of material, including biographies, 
work plans, and colleagues’ observations, have been used in these studies. 
And the findings emerging from these case studies have been confirmed in 
subsequent empirical, experimental studies. 

Nonetheless, it should be recognized that these case studies have 
focused on elements or aspects of expert performance; they have not 
examined expert performance within a complex, holistic, domain. One such 
domain where expert performance is valued is leadership (Mumford et al., 
2009). The evidence accrued in various studies of leadership points to the 
value of case studies in understanding expert performance by leaders — a 
distinctly complex form of performance. 

Leaders provide, or leave behind, a rich body of archival material. 
Moreover, the attention we give to leaders ensures a large body of 
observations are available to appraise leader performance. As a result, studies 
of leadership have long used case studies to both develop and test various 
theories (Bligh & Hess, 2007; Maslin-Wicks, 2007). The routine use of case 


studies in leadership research has, moreover, resulted in rather disciplined 


methods for conducting such studies. Thus while case studies might be 
employed in studying other forms of social expertise (Jiang & Thagard, 


2014), we will, in the following, focus primarily on leadership studies. 


In a single case study, Friedrich et al. (2014) studied George C. 
Marshall, Army Chief of Staff and Secretary of State, in an attempt to test 
Friedrich, Vessey, Schuelke, Ruark, and Mumford’s (2009) theory of 
collective leadership. As Yammarino, Salas, Serban, Shirreffs, and Shuffler 
(2012) have pointed out, at its heart, this theory holds that effective 
leadership is based on viable, timely exchange of information between and 
among team members with respect to the leader. Thus this theoretical model 
stresses the importance of social networks, communication, and expertise 
based on delegation when followers possess unique, critical, expertise for 
mission execution. 

To test this theoretical model, Friedrich et al. (2014) reviewed seven 
academic biographies of Marshall. After ensuring the validity of biographies 
through systematic review of the accuracy and detail of material presented, 
collectivistic leadership events were identified based on criteria specified by 
Yammarino et al. (2010a, 2010b). Following elimination of events 
conflicting across biographies, 102 unique collective leadership events were 
identified. These events were treated as the key unit of analysis with event 
descriptions, drawn from the source providing the most detail, ranging from 
one paragraph to 16 pages in length with events spanning a 50-year career. 

Each event was appraised for the quality, creativity, and efficiency of 
the solution eventually used to address the problem broached by the 
leadership event. Judges rated each incident for viability of information 
networks, effective communication, and manifest leader skills. In keeping 
with Friedrich et al.’s (2009) model, it was found that communication 
network and leader skills were strongly related to collective actions on the 
part of teams (e.g. delegation, voice, empowerment) which, in turn, were 


found to determine the outcomes of problem solutions in a structural 


equations model. Thus cases do not prohibit statistical analysis when the 
population of events is adequate for such analyses. 

Not only have studies of single cases been employed in studies of 
leadership, but studies of multiple cases are also commonly conducted. In one 
effort along these lines, Mumford (2006) proposed a theory of leadership 
style which held that approaches to leadership problems were based on how 
leaders appraised causes (e.g. locus of causation, controllability of causes, 
origin of causes) and outcomes sought through manipulation of these causes 
(e.g. time frame, number of outcomes, nature of outcomes). Patterning of 
these variables has been used to develop the notion of charismatic (future 
focused, people as causes), ideological (past focused, situational variables as 
causes), and pragmatic (present focused, both people and situational variables 
as causes) leadership styles. Mumford identified 40 historically notable 
charismatic (e.g. George W. Bush), ideological (e.g. Ronald Reagan), and 
pragmatic (e.g. Eisenhower) leaders, half of whom were socialized (e.g. 
Roosevelt) and half personalized (e.g. Hitler). 

Viable, positively reviewed, academic biographies were obtained for 
each leader. Summary chapters were used by three judges to code criteria. 
Other panels consisting of three to five judges reviewed “in power” chapters 
and “rise to power” chapters to code multiple variables bearing on problem- 
solving (e.g. information gathering, solution evaluation), social exchange 
relationships (e.g. autonomy, collaboration), communication styles (e.g. 
audience, confrontation, elegance), and political tactics (e.g. bargaining, 
coalition formation). These panels of judges displayed excellent agreement. 
Discriminant analyses were used to contrast six types of leaders described by 
the CIP framework. In this framework, leaders are labeled as either 


personalized or socialized based on the types of goals the leader is attempting 


to achieve. Furthermore, leaders are also categorized as_ charismatic, 
ideological, or pragmatic based on the mental models they convey to 
followers. Thus, according to this framework, leaders are categorized based 
on both their goal orientation and mental models, such that the two categories 
are crossed to create a total of six leader types. It was found that charismatic, 
ideological, and pragmatic leaders displayed meaningful differences even 
when controls were taken into account. This controlled, comparison group 
approach revealed ideological leaders maintained a close cadre of followers, 
while exhibiting a creative, albeit scholastic, problem-solving style. Other 
research using historical case data by Bedell-Avers et al. (2009), Hunter, 
Cushenbery, Thoroughgood, Johnson, and Ligon (2011), Mumford, Antes, 
Caughron, and Friedrich (2008), and Mumford et al. (2007) has provided 


support for the conclusions flowing from this model. 


Experimental Studies of Leader Styles 


Our foregoing observations point to the value of case studies when source 
material is well defined, controls are employed, and judges are given 
adequate training — in all these studies judges were given extensive training 
and appropriate comparison groups are specified (Parry et al., 2014). 
Nonetheless, the question remains as to whether these findings are confirmed 
in other studies using other methods. An initial answer to this question has 
been provided by Bedell-Avers, Hunter, and Mumford (2008). They 
developed a measure of people’s preferred leadership style. This measure 
presented abstracts, one paragraph, of a charismatic, ideological, and 
pragmatic leader’s speech. People were asked to indicate which speech they 
believed was most similar to them. The 12 speech sets yielded reliable (r,, = 
0.70) assessments of leader style. This measure was administered to a sample 
of undergraduates who were asked to provide solutions to academic and 
social leadership problems involving consideration, initiating structure, 
participation, and change management where problem solutions were 
appraised by judges for quality and originality. It was found that charismatic, 
ideological, and pragmatic leaders differed in the type of problems and the 
domain of problems they were successful in solving. 

In another study along these lines, Hunter, Bedell-Avers, and Mumford 
(2009) administered the Bedell-Avers et al. (2008) measure to a sample of 
undergraduate students. After classifying participants as evidencing a 
charismatic, ideological, or pragmatic style, they were asked to work on a 
university leadership simulation exercise where performance was assessed 


using the outcomes of the simulation game. Manipulations were made in the 


complexity of the simulation exercises and the time frame participants should 
employ in thinking about how to perform. It was found that different leader 
types performed better under certain conditions of complexity and in certain 
time frames for thinking about performance. Thus, charismatic leaders 
performed well in low complexity situations when they were asked to think 
about the future, while ideological leaders performed well in complex 


situations where they were asked to think about the present. 


Expert Technical Performance 


Cases of Technical Performance 


Of course, leaders are not the only profession where rich historical records 
are available for case studies. Scientists also leave extensive records of their 
work. And, as a result, case studies have been used to assess expert 


performance among scientists and engineers (Weber & Perkins, 1992). In one 


study along these lines, Crouch (1992) conducted a case study examining the 
Wright brothers’ development of powered flight. This case study was based 
on historical records. Review of these records led to the conclusion that much 
of the material needed for powered flight, principles of lift and the wind 
tunnel, had been developed earlier. More centrally, due to their prior work as 
bicycle mechanics, they introduced the principle of balance into their design. 
And, with introduction of this fringe principle, powered flight became 
possible. 

The importance of expertise, and fringe search, has been demonstrated 
in a recent study of multiple cases of performance in the sciences and 
engineering conducted by Barrett et al. (2014). They obtained favorably 
reviewed academic biographies of 93 scientists, written after 1950. Each 
biography was reviewed by two doctoral students, evidencing inter-judge 
agreement of 0.78, to abstract passages dealing with adversity, work 
collaborations, and work strategies. Judges reviewed these biographies to rate 
markers of scientific influence on individuals, groups, organizations, and the 
field. In addition to these subjective measures of scientific performance, 
objective measures were obtained including the scientist’s h-index, the 
number of creative products produced, and the number of teams led. A 


subsequent factoring of these criteria yielded five dimensions: (1) social 


influence, (2) theoretical influence, (3) technical influence, (4) groups led, 
and (5) professional influence. 

To predict these criteria, nine trained judges working in three-person 
teams evidencing inter-judge agreement coefficients above 0.70, were asked 
to read abstracted material for adversity, collaboration, and work strategies. 
In each area, they were asked to rate the material presented on a set of 
benchmark rating scales. Thus adversity was appraised with respect to 
adverse events, coping strategies, motivation, and opportunity search; 
collaborations were appraised with respect to professional activity, 
professional relationships, opportunities for collaboration, active 
collaboration, and participation; and work strategies were appraised with 
respect to prestige, network, area knowledge, general strategies, core 
research, fringe research, and specific strategies. In addition, another panel of 
judges appraised relevant biographical controls, such as strength of 
documentation, length of biography, factual focus, and field versus lab work. 

When the five criteria were regressed on these predictors, a rather clear 
set of conclusions emerged. First, adversity was not a strong, or significant, 
predictor of scientific performance on any of the five criteria. Second, both 
theoretical and technical performance could be positively predicted by 
collaboration strategies and work strategies. Third, participation predicted 
theoretical influence. Fourth, area expertise and fringe research were found to 
be strongly positively related to both theoretical and technical influence. Thus 
this multiple case study confirmed Crouch’s (1992) observations concerning 


expertise and fringe research in scientific performance. 


Experimental Studies of Innovation 


Although a number of experimental studies could confirm the role of 


expertise in the technical and theoretical performance of scientists (Chi, 


Feltovich, & Glaser, 1981), perhaps the most direct evidence in this regard is 
provided in a study by Mumford, Baughman, Supinski, and Maher (1996) 
examining creative problem-solving — the basis for technical and theoretical 
performance among scientists. In this study, undergraduate participants were 
asked to provide written solutions to three issues involved in solving a 
creative marketing problem and three issues involved in solving creative 
public policy problems. Judges rated the quality and originality of the 
solutions provided to each issue evidencing adequate (r,, = 0.70) inter-judge 
agreement. 

Prior to working on these problems, participants were asked to work on 
an information gathering task. This task presented four problems, two drawn 
from the managerial domain and two drawn from the public policy domain. 
Following presentation of each problem, participants were presented with 
different types of information where time spent reading each type of 
information was recorded. The seven pieces of information, each presented 
on a different screen, examined (1) key facts, (2) information bearing on 
goals, (3) information bearing on restrictions, (4) information bearing on 
principles, (5) diverse information, (6) anomalous information, and (7) 
irrelevant information. It was found that the undergraduates producing the 
highest quality and most original solutions to these creative problems spent 
more time reviewing key facts and anomalies. Thus this experimental study 


confirms the findings emerging in Crouch’s (1992) single case study of 


scientific performance and Barrett et al.’s (2014) study of multiple cases of 
scientific performance. As a result, it appears that cases can be used to attain 
an understanding not only of leader performance, but also of scientific 
performance and, more broadly, the creative thinking on which scientific 


performance is based. 


Contingencies in Expert Performance 


Contingencies in Expert Performance Case Studies 


Our foregoing observations indicate that systematic case studies can be used 
to identify skills contributing to expert performance — specifically forecasting 
and causal analysis skills. Moreover, the importance of these skills was 
confirmed in subsequent experimental studies. However, these studies were 
based on one source of case data, historical material, as analyzed primarily by 
one investigator, Mumford and colleagues (Mumford, 2002; Mumford & 


Moretz, 2003), using a purely theoretical approach. Thus, a critical question 


emerges: can other source material, or cases, be used by other investigations 
to draw conclusions about expert performance? An initial answer to this 
question has been provided by studies examining how external contingencies 
— constraints — influence expert performance. 

In one study along these lines, Weisberg (2011) studied a single case of 
expert performance. This was Frank Lloyd Wright’s design of a historically 
notable building — Falling Water. In this analysis, a variety of observations 
bearing on Wright’s work on this project were examined, including 
interviews, architectural histories, and Wright’s own writings. Of special 
significance, however, was the use of work plans as a basis for this analysis. 
Weisberg’s analysis of this material led to two noteworthy conclusions. First, 
expert achievement is, in part, based on past projects with key elements 
embedded in past problem solutions being used to address a new problem. 
Thus Prairie Houses appeared to contribute to the use of cantilevered 
balconies while Taliesin, Wright’s studio, appeared to contribute to the use of 
a stone core. Second, formulation of the plan for Falling Water was based, in 
part, on analysis of the specific constraints applying in the situation at hand. 


Thus site constraints vis-a-vis the principle of maximizing sunlight led 


Wright to place the house over the waterfall. Other work by Rich and 
Weisberg (2004) examining another historically noteworthy work product, 
the development of the television show All in the Family, also points to the 
expert’s attention to contextual constraints in the development of viable 
innovative products. 

Although Weisberg and colleagues (e.g. Ramey & Weisberg, 2004; Rich 
& Weisberg, 2004; Weisberg, 2011) have examined the analysis of work 
products vis-a-vis single notable production, other scholars have used 
multiple case studies to arrive at similar conclusions. Stokes and Fisher 
(2005) examined the body of artistic work produced by Max Beckmann and 
Philip Guston — two twentieth-century artists. They concluded, based on 
studies of these two cases, that goal, source, and task constraints imposed by 
artists on their work were central to their production of noteworthy, expert, 
artworks. Indeed, changes in the constraints imposed by these artists led to 
subsequent shifts in their artistic style and the production of qualitatively 
different, albeit still noteworthy, pieces of artwork. Other case studies of 
artists by Stokes (2008, 2009) provide support for this conclusion. More 
centrally, these studies of multiple artists’ products, like Weisberg’s (2011) 
studies of single historically notable cases, point to the impact of constraint 


analysis and manipulation on expert performance. 


Experimental Studies of Constraints 


These case studies are noteworthy, in part, because they point to the value of 
using work products as a basis for case studies. More centrally, the findings 
obtained in these studies suggest that experts, in performance, work by acting 
within relevant constraints. Some initial experimental studies pointing to the 
appropriateness of this conclusion have been provided in a series of three 
experimental studies, where undergraduates were asked to work on creative 


problem-solving tasks where constraints were made more or less restrictive 


(e.g. Baughman & Mumford, 1995). It was found that less restrictive 
constraints resulted in more viable, or original, products when the constraints 
were present. When more restrictive constraints were presented, originality 
increased when the constraints were related. 

Recently, Medeiros, Partlow, and Mumford (2014) have provided an 
experimental study examining the impact of constraints on a “real-world” 
creative problem-solving task. Here, study participants, undergraduates, were 
asked to formulate marketing campaigns for a new product — a high energy 
root beer. The resulting marketing plans were appraised for quality, 
originality, and elegance. Emails sent by the participant’s putative supervisor, 
the Vice President of Sales, prior to participants starting work on their 
advertising plans, were used to establish constraints. Based on earlier 
observations bearing on the nature of “real-world” task constraints (e.g. 


Mumford, Peterson, & Robledo, 2013), constraints were specified with 


respect to marketing fundamentals, marketing themes, available information, 
and resources in a fully crossed design. It was found that when constraints 


presented were not malleable, for example resource constraints, products of 


lower quality, originality, and elegance were obtained. When, however, the 
constraints presented could be manipulated, for example, information 
constraints, solutions of higher quality, originality, and elegance were 
obtained when people needed to think about these constraints and how they 


might be manipulated. 


Conclusions 


When one considers the material presented in this chapter, a reevaluation of 
the methods employed in studies of expertise and expert performance seems 
indicated. Traditionally, scholars have viewed case studies with suspicion. 
Indeed, they are often used simply to tie findings emerging from other 
methods — experimental, psychometric, field studies — to incidents of “real- 
world” performance. The key conclusion emerging from the present chapter, 
however, is that case studies can, and should, play a more fundamental and 
prominent role in studies of expertise and expert performance. 

We have shown that case studies can be used to identify key capacities 
underlying expert performance capacities such as forecasting and causal 
analysis (Marcy & Mumford, 2007; Mumford, 2002; Shipman et al., 2010). 


We have shown that case studies can be used to identify key variables 


shaping expert performance such as the identification and manipulation of 
constraints (Medeiros et al., 2014; Weisberg, 2011). We have shown that case 
studies can be used to understand expert performance in complex social 
situations (Hunter et al., 2009; Mumford, 2006). And we have shown they 
can be used to understand complex technical performance (Crouch, 1992; 
Mumford et al., 1996). Not only did these case studies lead to viable 
conclusions about expert performance, the key conclusions emerging from 
these case studies, both studies of a single case and studies of multiple cases, 
were confirmed in subsequent experimental investigations. 

Our foregoing observation is of some importance because it implies that 


case studies do lead to conclusions which triangulate — at least in the case of 


expert performance (Parry et al., 2014). The conclusion, however, points to a 
broader question — why do case studies work well, really exceptionally well, 
in studies of expert performance? In this regard, it is important to bear in 
mind one key consideration that gave rise to initial criticisms of case studies. 
Often these studies were used to inform underlying, invisible, motives. And, 
indeed, under these conditions, case studies are suspect. However, in studies 
of expertise and expert performance, case studies are based on observations — 
sometimes behavioral observations validated by authors, as is the case in 
biographical studies (Mumford, 2006), sometimes actual work products or 
work performance (Weisberg, 2011). Observations of concrete observable 
behavior, or work products, do provide a plausible basis for conducting case 
studies. 

Experts, exceptional high-level experts, moreover, attract attention from 
multiple observers. Thus, their behavior is observed, recorded, and 
remembered by others. Experts, furthermore, leave behind their work plans. 
And finally, the outcomes of their plans and behavior over time, often long 
periods of time, can be appraised (Parry et al., 2014). As a result, case studies 
may be a particularly appropriate vehicle for the study of expert performance. 

By the same token, a key limitation of these studies is evident in the 
present effort. Typically, case studies, either single unit or multiple unit case 
studies, will be based on observations or records of exceptional high-level 
performers. Thus, case studies’ greatest utility will be in appraising the 
performance of truly exceptional experts. Less exceptional people leave less 
of a record, if any, behind. As a result, the question remains as to whether the 
conclusions flowing from case studies can be generalized to less exceptional 
people. In point of fact, this limitation is why case studies should be, indeed 


we argue must be, accompanied by experimental, psychometric, and field 


studies — at least when general conclusions, conclusions not bound to tally 
exceptional levels of performance, are to be drawn. 

Another limitation inherent in case studies pertains to the number and 
nature of the cases which provide a basis for drawing inferences about expert 
performance (Parry et al., 2014). Clearly, in the present investigation, single 
case studies (Mumford, 2002; Weisberg, 2011) led to conclusions consistent 
with those emerging from multiple case studies (Mumford & Moertl, 2003; 
Stokes & Fisher, 2005). Nonetheless, as the number of cases increases, or the 
number of events drawn from a given case increases, so too does the strength 
of the inferences that may be drawn about expert performance. Indeed when 
enough events can be identified, even a single case may provide the base for 
advanced multivariate analysis. In fact, Barrett et al.’s (2014) study of 
collective leadership on the part of George Marshall provides an ideal 
illustration of this potential in case studies. Remember the key is not the 
number of participants, but the number of events being analyzed that permits 
application of advanced analytic procedures — in other words, the level of 
analysis may be events not people (Dinh et al., 2014). 

The strength of the inferences permitted by case studies also clearly 
depends on the quality of the method used. Were multiple cases studied? Was 
the source material for each case extensive, verified, relevant, and 
systematically sampled? Were multiple judges, suitably trained, used to 
appraise case material? Were appropriate controls relevant to both the 
phenomena of interest and the case material being appraised actually 
employed? Were comparison groups (Mumford, 2006) or comparison 


theories (Barrett et al., 2014) actually appraised and tested? If these, and a 


number of other methodological issues noted earlier, are actually addressed, 


then case studies may provide a strong, not weak, basis for the study of 
expert performance. 

Bearing these limitations in mind, future research has a wide range of 
opportunities to pursue in this area. Potential areas for future research include 
using cases to gain a more concrete and detailed understanding of, not only 
expert performance, but other areas such as creativity and leadership as well. 
More specific to the context of expert performance, future research should 
explore contemporary case data, such as coaches of sports teams and 
Michelin star chefs. Past research has successfully utilized modern 
historiometric data to analyze complex subject matter, such as leadership 
(Hunter et al., 2011). Objective archival data on sports coaches are available 
and provide rich information about their success and coaching style. 
Moreover, coaching statistics, such as the number of games won or number 
of playoff games attended, are easily accessible. In addition to reviews and 
menus available online, case studies can utilize interviews, observations, and 
food tastings to garner information about expertise of professionally trained 
chefs. Recent research has utilized this content domain to explore topics such 
as creativity and intuition (Stierand & Dorfler, 2016). An additional avenue 
for future research is examining less exceptional individuals, or “non- 
experts,” using case studies. Given the enhancements in technology and 
social media, a more thorough record of less exceptional individuals is being 
left behind. By conducting case studies on average individuals, conclusions 
emanating from case studies can be generalized to less exceptional people. 

Indeed, we believe there is much to recommend the use of case studies 
in attempts to understand expert performance. Cases come from the “real 
world.” As a result, they may have greater generality than many more 


traditional studies. Cases, moreover, allow multiple outcomes or criteria to 


unfold over long periods of time. Thus our studies are not limited to 
immediate, short-term, indices of performance. Cases, moreover, look at 
performance in a “real-world” context. The expert performance can be 
studied in situ and the effects of context on performance examined. Given 
these desirable qualities of case studies, we believe there is much to 
recommend their application in research on expertise and expert performance. 
We hope the present effort serves as impetus for future research on expertise 
and expert performance where cases and case study materials are more 


commonly employed. 
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Introduction 


Of the many methods applicable to the scientific study of expertise and 
expert performance, historiometrics is perhaps the least well known and least 
frequently used. Therefore, before I can discuss the technique any further it 
must first be defined. According to the only monograph devoted specifically 
to the subject, “historiometrics is a scientific discipline in which nomothetic 
hypotheses about human behavior are tested by applying quantitative 
analyses to data concerning historical individuals” (Simonton, 1990, p. 3). 


This definition contains three central concepts: 


1. Historical individuals are persons who have “made a name for 
themselves” or who have “left a mark on history” by some superlative 
achievement. Possibilities include recipients of the Nobel Prize, 
politicians elected President of the United States, world chess 
champions, and athletes who have won medals in the Olympics. It is this 
feature of historiometrics that makes it ideally suited for the study of 
expert performance. After all, such accomplishments are presumed to 
require a high degree of expertise, and when expert performance attains 
world-class levels in many domains, the result will be awards, honors, 
and other forms of recognition (Simonton, 2014b). Of course, the 
adjective “historical” actually assumes an underlying dimension that is 


quantitative rather than qualitative (Simonton, 1990). An athlete who 


wins a gold medal in the Olympics represents a higher degree of 
achievement than one who is a national champion, just as the national 


champion represents a degree above an athlete with even more local 


eminence. Moreover, within each of these groups athletes can be 
differentiated according to whether they ranked first (gold), second 
(silver), third (bronze), or even lower down in ordinal position. In fact, 
in some domains of achievement, such as tennis and chess, objective 
ranking systems exist that place the leading competitors along an ordinal 
or interval scale (e.g. Elo, 1986; Schulz & Curnow, 1988). 


2. Quantitative analyses consist of two features. First, historiometrics 
requires objective measurement of well-defined variables along a 
nominal, ordinal, interval, or ratio scale. In this sense, historiometrics 
does not differ from psychometrics except that the measurement 
techniques are applied to historical individuals. Second, the 
measurements are subjected to statistical analyses using the full panoply 
of tools available for drawing inferences from correlational data. These 
two features set historiometrics apart from psychobiography and 
psychohistory, an approach to the psychological study of historical 
figures that entails qualitative rather than quantitative analysis (Schultz, 
2014). 


3. Nomothetic hypotheses concern general laws or regularities of human 
behavior. For example, a considerable research literature has grown 
around whether expert performance in various domains is described by a 
distinctive age curve (Simonton, 1988a, 2012). Thus in the case of 
creative expertise, some have argued for a positive monotonic function 


resembling a learning curve (Ohlsson, 1992; Stroebe, 2010) whereas 


others have proposed a non-monotonic, single-peaked function with a 


mid-career optimum (Lehman, 1953; Simonton, 1997). This aspect of 


historiometrics constitutes another characteristic that separates this 


method from psychobiography and psychohistory. The latter approach 
favors the idiographic rather than the nomothetic, that is, it attempts to 
explain the distinctive attributes of eminent personalities. I should also 
point out that the historiometric emphasis on the nomothetic almost 
invariably requires that the hypotheses be tested on large samples of 
historical individuals. Only in this way can the investigator be confident 
that the findings extend beyond the idiosyncrasies of any single research 
subject. As a consequence, single-case or N = 1 studies are rare in 


historiometric research (but see Simonton, 1986b, 2015b, 2015c), 


Now that the method has been defined, I would like to accomplish three tasks 
in the remainder of this chapter. First, I provide a brief history of the method. 
Second, I offer an overview of the diverse methodological issues involved in 
carrying out historiometric research. Third, I will review the main empirical 
findings that this approach has obtained with respect to expertise acquisition 


and expert performance. 


History 


Although historiometrics is not as well known as other techniques, it can be 
considered the first scientific method that was applied to the objective and 
quantitative study of expertise and expert performance. To be specific, the 
first bona fide historiometric investigation was conducted by Adophe 
Quételet back in 1835. Quételet is best known for his pioneering applications 
of statistics and probability theory to social phenomena. Much of this work 
concentrated on establishing the normal curve as descriptive of the 
distribution of human traits around some central value representing the 
homme moyen (or “average person’). Yet his empirical investigations were 
by no means confined to establishing the ubiquity of this symmetric 
distribution. He was also intrigued with the question of how creative 
productivity is distributed across the career. To address this issue, he 
scrutinized the lifetime output of eminent French and English dramatists. By 
tabulating their dramatic works into consecutive age periods, he was able to 
discern the characteristic longitudinal distribution. In addition, Quételet 
directly examined the empirical relation between quantity and quality of 
creative output. Not only was he the first to apply quantitative and objective 
techniques to biographical and historical data, but he also did so with a 
methodological sophistication that was not to be surpassed for nearly a 
century (Simonton, 2012). 

Unfortunately, Quételet’s (1835) historiometric inquiry was buried in a 
larger published work dealing with different topics, and so this particular 


contribution was largely ignored (Simonton, 1988a). Therefore, the first 


behavioral scientist to publish a truly influential historiometric study was not 
Quételet but rather Francis Galton. The specific work was Hereditary genius: 
An inquiry into its laws and consequences, which was published in 1869, 
albeit as a significant expansion of an earlier historiometric study published 
as an article four years earlier (Galton, 1865), The main goal of Galton’s 
investigation was to establish a biological basis for natural ability, the 
capacity underlying exceptional achievement in all its forms, including 
creativity, leadership, and sports. To accomplish this task, he collected 
extensive biographical data on the family pedigrees of eminent creators, 
leaders, and athletes, and then subjected these data to quantitative analysis. 
Unlike Quételet’s study, Galton’s investigation had both a short- and a long- 
term impact, thereby becoming one of the classics in psychology. In the short 
term, the work sparked a controversy among his contemporaries (e.g. 


Candolle, 1873) that led Galton (1874) to formulate the nature—nurture issue, 


one of the critical questions concerning the development of expertise 
(Ericsson, 2014; Simonton, 1999b). In the long term, the work inspired other 
historiometric inquiries into the role of genetics in exceptional achievement 
(e.g. Bramwell, 1948; Simonton, 1983). In addition, Galton’s research 
provided the impetus for James McKeen Cattell’s innovations in the area of 
the measurement of differential expertise, as gauged by achieved eminence in 
a domain (e.g. Cattell, 1903; see also Murray, 2003; Whipple, 2004). 

Despite the fact that the research conducted by Quételet, Galton, and 


Cattell was clearly historiometric, it was not identified as such by these or 
any other researcher at the time. Instead, the investigations were labeled as 
“empirical,” “scientific,” or “quantitative.” The method was not actually 
given a formal name until 1909, when Frederick Woods published an article 


in Science on “A new name for a new science.” There he defined the 


technique as encompassing those investigations in which “the facts of history 
of a personal nature have been subjected to statistical analysis by some more 
or less objective method” (p. 703). This definition was followed by a 1911 
article in the same journal on “Historiometry as an exact science” in which he 
claimed that the approach has some special value for research on the 
“psychology of genius.” Somewhat surprisingly, Woods’s own historiometric 
inquiries seldom dealt with this issue directly. Instead, his most important 
publications using this method were on the inheritance of intelligence and 


morality in royalty (Woods, 1906) and the influence of monarchs on their 


nation’s welfare (Woods, 1913). Hence, subsequent historiometric research 
most germane to expertise and expert performance was conducted by others. 
In this later work one investigation stands out well above the others: 
Catharine Cox’s (1926) The early mental traits of three hundred geniuses. 
This study forms the second volume of Terman’s (1925-59) classic work 
Genetic studies of genius. Although the other four volumes concern a 
longitudinal study of over a thousand intellectually gifted children, Cox’s is 
retrospective. Rather than collect data on gifted children and follow them into 
adulthood to see if they displayed world-class expertise, Cox gathered a 
sample of unquestionable geniuses — Napoleon, Luther, Newton, Descartes, 
Voltaire, Michelangelo, Beethoven, etc. — to determine whether they showed 
any signs of precocious intellect in childhood. After using biographies to 
compile a chronological list of early intellectual achievements, and applying 
the operational definition of the intelligence quotient as the ratio of mental to 
chronological age (times 100), she was able to obtain reasonably reliable 
estimates of IQ scores for nearly all those sampled. Significantly, Cox 


identified her study as an example of historiometrics. Not only was it an 


example, but it soon became an exemplar of the technique (cf. Simonton, 
2008a; Simonton & Song, 2009). 

In fact, Cox’s (1926) publication represents the climax of the early 
period of historiometric research. Subsequent investigations were seldom as 
ambitious, and few came anywhere close to the same level of methodological 
sophistication (see e.g. Raskin, 1936). The only work to come close to the 
same level was Harvey Lehman’s (1953) book Age and achievement which 
dealt with the same issue first investigated by Quételet (1835) over a century 
earlier. Nonetheless, historiometrics underwent something of a revival in the 
1960s and 1970s. As a consequence, the first book summarizing 


historiometric findings with respect to genius, creativity, and leadership came 


out in 1984 (Simonton, 1984c) and in 1990 appeared the first book totally 
devoted to explicating the methodological issues entailed in historiometric 


research (Simonton, 1990). Other publications on historiometric methods 


have appeared in Psychological Methods (Simonton, 1999a), the Annual 


Review of Psychology (Simonton, 2003b), and recent handbook chapters (e.g. 


Simonton, 2014b, 2015a), suggesting that the approach has become an 


accepted even if relatively rare methodology in psychological science. 


Methodological Issues 


As a scientific technique, historiometrics departs appreciably from other 
methods in the behavioral sciences (Simonton, 1999a, 2003b, 2015a). It 
certainly differs from experimental approaches, whether laboratory or field, 
insofar as it depends on correlational data analyses. In this sense it has a close 
affinity with psychometrics. Even so, historiometrics and psychometrics 
dramatically differ on several key methodological parameters. As a result, it 
is necessary to treat some of the technical concerns that are especially 
prominent in this distinctive approach (see also Simonton, 1990). These 
issues pertain to sampling procedures, variable definitions, research designs, 


and methodological artifacts. 


Sampling Procedures 


Most psychological research relies on research participants who are 
anonymous and inherently replaceable. This is especially the case for 
investigations that draw their samples from college undergraduates who sign 
up as research participants in order to fulfill a course requirement. The 
specific identity of the participant is not relevant, and one participant is 
presumed to be essentially equivalent to any other. Historiometric research, in 
contrast, depends on what has been called significant samples (Simonton, 
1999a). In this case, the individuals in the studies have known identities, and 
their identities are such that they cannot be said to be interchangeable with 
other participants. In particular, the participants are persons who have “made 
a name for themselves” by attaining eminence in some domain that presumes 
special expertise. Examples include famous or at least world-class creators, 
leaders, athletes, and performers (e.g. Kozbelt, 2008; Simonton, 2014c; 
Sulloway & Zweigenhaft, 2010; Wai, 2014; Zusne, 1976). Moreover, these 


luminaries have attained sufficient distinction to have substantial information 
about them readily available in archival sources, such as encyclopedias, 
histories, biographical dictionaries, autobiographies, and _ biographies. 
Accordingly, unlike what holds for any other general research method, 
historiometric samples can include individuals who are deceased. Indeed, it is 
not uncommon for a historiometric investigation to be confined to eminent 
achievers who have already finished their lifespans (e.g. Kozbelt, 2014; 
McKay & Kaufman, 2014; Simonton, 1975a). This capacity has critical 


implications for the study of expertise acquisition and expert performance 


because it becomes thereby possible to examine exceptional achievement 
across the entire life, from birth to death. It should be noted, too, that because 
the samples often consist of deceased celebrities, they cannot properly be 
called “participants,” as is the current convention, but rather they must be 
referred to by the older term “subjects” (Simonton, 1999a), 

Given the distinctive nature of historiometric samples, the next question 
is how to assemble the individuals who will become the research subjects. 
Sometimes the sample will be defined according to membership in well- 
defined groups of eminent achievers, such as all Nobel laureates in the 
sciences (e.g. Jones, Reedy, & Weinberg, 2014; Manniche & Falk, 1957; 
Stephan & Levin, 1993). The only limitation may be that some subjects will 
have to be deleted owing to the lack of necessary biographical data (see e.g. 
Cox, 1926). Other times the domain of achievement is not so specifically 
defined, such as the expert performance displayed by “great generals,” 
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“illustrious scientists,” “outstanding artists,” or “famous composers.” In such 
instances eligibility for the sample is more open ended. The most common 
procedure in this case is to sample those individuals who attain the highest 
degree of eminence in the targeted domain of achievement (e.g. Simonton, 
sample was derived from the most eminent historical personalities on 
Cattell’s (1903) list, where eminence was based on the amount of space 
devoted to each person in standard reference works (cf. Whipple, 2004). 
Because most domains of expertise are not well defined, sampling according 
to eminence is a very common procedure. However, it does have one major 
disadvantage: by selecting only those subjects who attain the highest degree 
of distinction, the investigator necessarily truncates the amount of variance 


that will be exhibited by many relevant variables. This variance truncation 


will often reduce the expected correlations that can be obtained between 


performance criteria and various predictor variables. 


Variable Definitions 


Historiometric inquiries into expertise generally must include two types of 
assessments. The first concern measures of actual performance and the 


second concern indicators of acquisition. 


Performance Measures 


Most commonly expertise is viewed as an attribute of individuals, and 
accordingly the assessment of expert performance is carried out at the level of 
individuals. In this case, individual attainment can be gauged in terms of (a) 
eminence as recorded by space allotted in reference works (Cox, 1926; 
Damian & Simonton, 2015; Simonton, 2014c; Whipple, 2004), (b) the receipt 


of major honors such as the Nobel Prize (Jones et al., 2014; Root-Bernstein et 


al., 2008), (c) total lifetime productivity or the output of highly influential 
works (Simonton, 1977b, 1991b, 1992b, 2015b), (d) objective scoring 


systems such as those used to rate chess players and athletes (e.g. Elo, 1986; 


Schulz & Curnow, 1988), (e) the attainment of high offices and positions, 
such as president, prime minister, pontiff, patriarch, or company CEO 
(Lehman, 1953; Simonton, 1984c, 1998; Wai, 2014; Wai & Rindermann, 


2015), and (f) subjective assessments based on surveys of scholars and other 


experts (Simonton, 1977a, 1992b, 2013). Occasionally, investigators have 
gauged historic individuals with respect to their display of multiple 
competencies, usually under the variable category of versatility (Cassandro, 
1998; Simonton, 1976a; Sulloway, 1996; White, 1931). 


However, sometimes historiometricians will adopt a more fine-grained 


analysis by taking particular achievements or events as the units of analysis. 
Performance is then gauged according to the differential impact or success of 
those units. Examples include the critical evaluations bestowed on motion 


pictures (e.g. Plucker, Holden, & Neustadter, 2008; Simonton, 2004b; Zickar 


& Slaughter, 1999), the frequency that an opera appears in the world’s major 


opera houses (Simonton, 2000), and whether a battle resulted in victory or 


defeat for a particular general (Simonton, 1980). Finally, sometimes the 
analysis of singular acts of exceptional performance will be aggregated into 
consecutive periods of a career, such as decades. For instance, investigators 
might examine how the magnitude of performance changes as a function of 
age (Kozbelt, 2014; Simonton, 1977a, 1985, 2015c). Alternatively, 
researchers might study how expertise in separate domains must be 
distributed across the career course so as to maximize impact or influence 


(Root-Bernstein, Bernstein, & Garnier, 1993; Simonton, 1992b). 


Acquisition Indicators 


Expert performance has numerous predictors, but certainly among the most 
crucial is the acquisition of the necessary competence in the first place. This 
acquisition has been accessed several ways. The easiest is to use an expert’s 
chronological age as a gauge of accumulated domain-specific experience. 
This is the measure used in the huge literature on the relation between age 


and exceptional achievement (Dennis, 1966; Lehman, 1953; Quételet, 1968; 


Simonton, 1988a). A more refined indicator is an individual’s career age, that 
is, the length of time that he or she has been actively engaged in making 
contributions to a given achievement domain (Kaufman & Kaufman, 2007; 
Simonton, 1988a, 1997, 1998). For example, in the sciences this may be 
defined as the years that have transpired since a person received their PhD. 
An even better measure for most purposes is the number of years that have 
transpired since an individual initiated formal training in the domain. Thus, 
expertise acquisition in classical composers has been assessed by the number 
of years that have elapsed since they first began music lessons (Hayes, 1989; 
Simonton, 1991b, 2000, 2016). Even more superior, perhaps, are studies that 
gauge acquisition according to the number of products or achievements 
within a domain, such as the number of films directed (Zickar & Slaughter, 


1999), the number of symphonies composed (Simonton, 1995, 2015b), and 


the battles fought or won (Simonton, 1980). Of course, all of these indicators 
are explicitly or implicitly temporal in nature. Therefore, sometimes 
historiometricians will assess other features of the expertise-acquisition 
process. For instance, an inquiry might focus on the influence of domain- 


specific mentors and role models, including both their number and their 


eminence (Simonton, 1977b, 1984a, 1992b, 1992c; Walberg, Rasher, & 
Parkerson, 1980). 


Research Designs 


Historiometric studies of expertise acquisition and expert performance have 
adopted a diversity of research designs. This diversity reflects the complexity 
of history-making achievements, a complexity that requires that the 
phenomenon be scrutinized from multiple perspectives. Nevertheless, most of 
the published research falls into one of the following three categories: cross- 


sectional, longitudinal, and mixed. 


Cross-Sectional Designs 


Expertise exists in degrees, and thus varies across individuals. At one 
extreme there are persons who are completely uninitiated in even the basic 
knowledge and skill, while at the other are individuals who display world- 
class competence — the recipients of prizes, medals, honors, and other forms 
of universal acclaim. Between these extremes are novices, who at least know 
the basics of the domain, and those persons who may attain professional 
status without reaching the highest levels of performance. Hence expertise 
can often be conceived along a quantitative scale that defines a dimension on 
which individuals may vary. The goal of cross-sectional designs is to 
discover the factors that are responsible for this substantial variation. Of 
course, the underlying factors are both numerous and diverse (Damian & 


Simonton, 2014; Simonton, 1987a). Yet certainly among the most critical are 


those factors that pertain to the acquisition of domain-relevant knowledge 
and skill. Hence, historiometricians have used cross-sectional designs to 
assess the following: (a) the eminence attained by artists or scientists as a 
function of the number and distinction of their teachers and mentors 
(Simonton, 1984a, 1992b, 1992c), (b) the probability of a general winning a 
battle as influenced by the amount of battle experience they have 


accumulated over the course of their military career (Simonton, 1980), (c) the 


magnitude of an opera’s success as determined by previous compositions in 


similar and dissimilar genres (Simonton, 2000), (d) the degree to which the 


performance of a US president is contingent on prior experiences, such as 
executive experience as a State governor, legislative experience in Congress, 


or military experience as an army general (Simonton, 2013). 


Longitudinal Designs 


An alternative procedure is to trace the course of expert performance across 
time. By conducting such a longitudinal analysis the investigator can trace 
the growth and decline in the capacity for exceptional achievement. This 
approach has two main forms: individual and aggregated. Individual designs 
scrutinize the performance of a single expert over the course of his or her 
career. An example is Ohlsson’s (1992) demonstration that Isaac Asimov’s 
output of books can be described according to a standard learning curve. 
Such single-case designs have also been applied to such historic figures as 
Napoleon (Simonton, 1979), Shakespeare (Simonton, 1986b), Beethoven 
(Simonton, 1987b, 2015b), Mozart (Kozbelt, 2005), and Edison (Lehman, 
1953; Simonton, 2015c). The obvious drawback to this approach is that the 


observed fluctuations in performance may be idiosyncratic to that particular 
person. There is no guarantee that the longitudinal results would generalize to 
a larger collection of experts drawn from the same domain. As a 
consequence, the vast majority of longitudinal studies utilize an aggregated 
design (Simonton, 1988a). In this case, performance is averaged across 
multiple experts, producing an overall career trajectory in which individual 
idiosyncrasies cancel out. The number of cases making up the aggregated 
analysis may run into the hundreds. This mode of analysis was first 
introduced by Quételet (1968) and was most extensively used by Lehman 
(1953). It is difficult to identify a form of world-class expertise to which this 
design has not been applied. Examples include creativity, leadership, sports, 


and chess. 


Mixed Designs 


Although aggregated longitudinal designs are widely used and have a long 
history, they suffer from a number of methodological problems (see below). 
AS a consequence, historiometricians have more recently applied mixed 
designs that integrate individual and aggregate levels of analysis. The first 
mixed design was cross-sectional time-series analysis. Here the performance 
data are tabulated across consecutive units of an individual’s career, 
producing individual-level time series, but then the age functions are 
estimated across multiple time series representing more than one career. This 


approach was first applied to the study of ten top classical composers 


(Simonton, 1977a), but was later applied to the careers of ten eminent 


psychologists (Simonton, 1985) as well as to the reigns of absolute monarchs 


in Europe (Simonton, 1984d) and Great Britain (Simonton, 1998). A far more 
sophisticated procedure takes advantage of the latest advances in multilevel 
designs. An excellent example is Zickar and Slaughter’s (1999) use of 
hierarchical linear modeling to assess the creative performance of 
distinguished film directors. The method permits the estimate of a typical 
career trajectory while at the same time obtaining estimates for each expert 


making up the sample (cf. Hass & Weisberg, 2015). 


Methodological Artifacts 


By the very nature of the method, historiometric research is correlational 
rather than experimental. As a necessary repercussion, such research lacks the 
random assignment to treatments required for the secure causal inferences 
found in laboratory and field experiments. Instead, controls must be 
implemented statistically, most often via a multiple regression analysis 
(Simonton, 1990). That is, in addition to the substantive variables that are 
directly relevant to the hypothesis at hand, the investigator must include one 
or more control variables that permit statistical adjustment for potential 
artifacts. In particular, statistical controls help the researcher avoid the 
intrusion of spurious associations. These controls include such variables as 
birth year, lifespan, gender, nationality, and domain of expertise. For 
instance, one historiometric inquiry examined the ages at which scientists 
produce their first major contribution, their single best contribution, and their 
last major contribution (Simonton, 1991a). The specific issue was whether 
the location of these three career landmarks varied according to the specific 
scientific discipline. However, such _ interdisciplinary contrasts are 
contaminated by the fact that life expectancies are not constant across 
domains. Mathematicians in particular tend to live less long than scientists in 
other disciplines. Accordingly, the interdisciplinary differences had to be 
estimated after first controlling for life expectancy. 

The specific source of spuriousness depends on the nature of the 
substantive question. What may be an essential control variable in one study 


may prove irrelevant in another. 


Empirical Findings 


Since Quételet’s (1968) pioneering study, historiometric research has come 
up with an impressive inventory of empirical results. These can be grouped 
into two categories, namely, those concerning the acquisition of expertise and 
those regarding expert performance. In each category I will begin by giving 
an overview of some of the central findings and then end by describing a 


particular historiometric study that addresses an issue in that category. 


Expertise Acquisition 


Overview 


Early historiometricians were often interested in the developmental 
antecedents of exceptional achievement (e.g. Bowerman, 1947; Candolle, 
1873; Cox, 1926; Ellis, 1926; Galton, 1869; Raskin, 1936). Indeed, one of the 
original arguments for the method was that it could provide important 
insights into the origins of genius and talent (Woods, 1911). Furthermore, 
many of the early empirical findings have been replicated and extended in 
more recent historiometric work (e.g. Damian & Simonton, 2015; Simonton, 
1976a, 1984a; Walberg et al., 1980). The most critical of these results 
concern the following three factors (Damian & Simonton, 2014; Simonton, 
1987a). 


First, world-class expertise tends to emerge from a distinctive family 


background. As already noted, Galton’s (1869) classic inquiry was dedicated 
to showing that eminent achievers tended to come from distinguished family 


pedigrees (see also Bramwell, 1948; Simonton, 1983). Although a significant 


portion of this tendency may reflect the influence of nurture rather than 
nature, the relevance of genetic endowment cannot be totally dismissed 
(Simonton, 1999b, 2014a). For instance, notable individuals in certain 
domains of achievement are also more prone to come from family lineages in 
which the incidence rate of psychopathology is above the population average 
(Karlson, 1970; see also Kyaga, 2015). Nevertheless, numerous 
historiometric investigations have documented how certain family 
circumstances serve as environmental factors that influence the acquisition of 
extraordinary expertise (Damian & Simonton, 2014; Simonton, 1987a). 
These factors include socio-economic class, early traumatic experiences, and 
birth order (e.g. Damian & Simonton, 2015; Raskin, 1936; Simonton, 2008b; 


Sulloway, 1996). Interestingly, these factors work primarily by channeling a 
young person into a particular form of expertise. For instance, scientific 
creators, in comparison to artistic creators, are more likely to grow up in 
stable and conventional homes with highly educated parents who pursue 
professional occupations (Simonton, 2004a). 

Second, genius and exceptional talent are associated with distinctive 
education and training. Galton’s (1869) assertion that exceptional 
achievement is born rather than made is plain wrong. The literature on 
expertise acquisition suggests that it requires around a decade of committed 


training and practice to attain world-class expertise (Ericsson, 2014), an idea 


that has received endorsement from historiometric research as well (Hayes, 
1989; Simonton, 1991b, 2016). This acquisition process can take a multitude 
of forms, including formal education, private instruction, coaching, or 
mentoring, exposure to domain-specific role models, and various forms of 
self-education, such as omnivorous reading (Raskin, 1936; Simonton, 1976a, 
1984a, 1986a, 1992a; Simonton & Ting, 2010; Walberg et al., 1980). 


Nonetheless, historiometric research also has pinpointed some important 


qualifications and complications regarding this “ten-year rule” (Simonton, 
1996a, 2000, 2014a; cf. Hambrick, Macnamara, Campitelli, Ullén, & Mosing, 
2016). To begin with, the specific nature of the instruction and training 
depends greatly on the type of expertise being acquired (Raskin, 1936; 
Simonton, 1986a). For example, outstanding leaders require different 
educational experiences than do exceptional creators, and, within eminent 
creators, distinguished scientists need distinct educational experiences than 
do illustrious artists (Simonton, 2004a). In fact, there is evidence that in some 
domains of achievement it is possible to have too much formal education or 


scholastic success to be successful (Simonton, 1976a, 1986a; cf. Simonton & 


Ting, 2010; Wai, 2014). This can be viewed as a form of “overtraining” 


(Simonton, 2000). In addition, substantial individual differences exist in the 


amount of time used to master the domain-specific skills and knowledge that 
is needed for exceptional accomplishments (Cox, 1926; Raskin, 1936; 
Simonton, 1991b, 1992a, 2014a; Simonton & Ting, 2010). In particular, 
those who attain the highest levels of achievement are more likely to have 
undergone expertise acquisition at an accelerated rate. 

Third and last, expertise of the highest order is most likely to appear in a 
particular socio-cultural context. This reality is indicated by the fact that 
genius and talent are not randomly distributed across space and time but 
rather tend to cluster into particular geographical locations (Candolle, 1873; 
Charness & Gerchak, 1996; Yuasa, 1974) and historical periods (Kroeber, 
1944; Murray, 2003; Simonton, 1988b, 1996b; Simonton & Ting, 2010). The 


underlying causes of such clustering involve a host of social, political, 


economic, and cultural factors (Simonton, 2003a). For instance, a large 
portion of the temporal clustering of exceptional creators and leaders can be 
attributed to the availability of domain-specific role models (Murray, 2003; 
Simonton, 1975b, 1988b, 1992a; Simonton & Ting, 2010). In particular, the 
number of great achievers in one generation is a positive function of the 
number in the preceding generation. In the specific area of creativity, some 
political environments tend to nurture creative development whereas others 
tend to discourage creative development (Simonton, 2003a). For example, 
exceptional creators are less likely to develop during times of political 
anarchy but are more likely to develop during periods of political 
fragmentation, when a civilization is divided into numerous independent 
states (Simonton, 1975b, 1976b; Simonton & Ting, 2010). Of course, another 


critical factor underlying the appearance of certain forms of high achievement 


is the value or importance that a particular culture assigns to that activity at a 


given point in time (Candolle, 1873; Charness & Gerchak, 1996; Murray, 


2003). Potential talent will not become fully realized in a milieu that 


discourages the corresponding domain of achievement. 


Illustration 


To provide a better idea of how historiometric investigations can contribute 
to our scientific understanding of expertise acquisition, I will describe in 


somewhat more detail a specific study in this area (Simonton, 1991a; see also 


Simonton, 2016). A major goal of the inquiry was to discern how individual 
differences in expertise acquisition are correlated with individual differences 
in expert performance. The particular domain under scrutiny was the 
composition of classical music. A sample of 120 eminent composers was 
obtained by taking those who had entries in two distinct reference works. The 
differential eminence of these composers was then assessed using six 
different sources, including their performance frequencies in concert halls, 
their ranking by musicologists, and the space devoted to them in various 
reference works. This composite eminence measure was shown to reflect a 
high degree of cross-cultural and transhistorical consensus regarding the 
creative achievements of these composers (see also Simonton, 1991c). Next, 
two sets of substantive variables were defined. 

First, for each composer were determined the total lifetime output, the 
maximum annual output rate, the age at maximum output, the age at first hit, 
the age at best hit, and the age at last hit, where a “hit” was a work that had 
obtained a secure place in the standard repertoire. These measures were all 
assessed two ways, namely, complete works (or compositions) and the 
individual themes (or melodies) making up those works. Two alternative 
definitions were used to take into consideration that works vary greatly in the 
magnitude of achievement, such as the contrast between a song and an opera. 


This contrast might be better captured by assessing the total amount of 


melodic material going into each work, a song having much fewer themes 
than an opera. 

Second, for each composer were gauged the age of first formal music 
lessons and the age at which composition began, including any juvenilia. 
These two measures were then combined with the assessment of age at first 
hit to create another set of variables: (a) musical preparation, or the age at 
first hit minus the age at first lessons and (b) compositional preparation, or 
the age at first hit minus the age at which composition was initiated. The first 
variable concerns how many years transpired between the onset of lessons 
and the first compositional success, whereas the second concerns how many 
years passed between the initiation of composition and the first success. 
Because age at first hit was defined two different ways (works and themes), 
there were actually two alternative indicators of musical and compositional 
preparation. 

Finally, two control variables were also included, namely, the 
composer’s birth year and the lifespan. The former allows adjustment for any 
historical trends whereas the latter permits adjustment for how long the 
composer lived, a factor that places an obvious constraint on lifetime output 
as well as the age at last hit. 

These measures were subjected to a series of correlation and regression 
analyses, with each analysis executed twice to confirm that the same results 
emerged for both themes and works. The analyses revealed that the onset of 
lessons and composition bore a prominent connection with expert 
performance. In the first place, the earlier a composer began music lessons, 
the sooner the first hit appeared, the higher the maximum output rate, and the 
higher the total lifetime output. The same pattern appeared for both works 


and themes. Thus, the most precocious and prolific composers tend to begin 


lessons and composition at relatively young ages. Even more striking were 
the results for the two preparation measures. Both were negatively correlated 
with maximum output rate, total lifetime output, and ultimate eminence as a 
classical composer. In other words, the greatest composers spend fewer years 
in music training and compositional practice before they started to make 
lasting contributions to the classical repertoire. The abbreviated preparation 
period is all the more remarkable given that the composers had begun 
expertise acquisition at a younger and thus, presumably, less mature 
chronological age. This finding implies that there may exist individual 
differences in musical talent that allow the most productive and eminent 
composers to accelerate expertise acquisition in their early developmental 
years. Lesser composers, in contrast, take a much longer time in music 
training and compositional practice before they can launch their creative 
careers (Simonton, 1996a). Yet, ironically, they not only must take longer, 
but begin later, too — prolonging the acquisition process all the more. 

This faster start for outstanding composers is not unique to classical 
music. The same pattern holds in other domains of creativity, such as the 
sciences (Simonton, 2002, 2004a). In addition, an early career 
commencement is associated with other key aspects of expert performance. 
Specifically, precocious impact is correlated with high annual productivity 
rates and total lifetime output (Lehman, 1946; Simonton, 1997). Hence, 
accelerated expertise acquisition is related with exceptional expert 


performance. 


Expert Performance 


Once an individual acquires his or her domain-specific expertise, how is that 
expertise manifested over the course of a career? More _ historiometric 
research has been dedicated to this question than to the issue of expertise 
acquisition. Again, I start with an overview of research findings and then tum 


to a specific illustration of the technique applied to this question. 


Overview 


The main thrust of research on this topic has been to determine the relation 
between age and outstanding achievement (Simonton, 1988a). As noted in 
the historical introduction, the first historiometric inquiry into this issue dates 


back to 1835 (Quételet, 1968). Since then, a host of age-performance studies 


have been published concerning various domains of world-class expertise, 
such as leadership (Oleszek, 1969; Simonton, 1984c, 1998), sports (Schulz & 
Curnow, 1988), and chess (Elo, 1965). Some of this diversity of domains is 


seen in Lehman’s (1953) classic Age and achievement. This compendium 
contains age-performance curves for achievements in the sciences, medicine, 
philosophy, music, literature, art, architecture, film, business, leadership, 
sports, and chess. Moreover, each of these areas of high accomplishment is 
usually broken down into numerous subdomains. For instance, the sports 
include baseball, football, ice-hockey, boxing, golf, tennis, car racing, 
billiards, bowling, and rifle/pistol shooting. Nonetheless, the vast majority of 
Lehman’s tables and graphs concern some form of creativity, an emphasis 


reflected in the general literature as well (Simonton, 1988a). Therefore, this 


brief overview will place the most stress on the key findings in the age 
function of world-class creative performance. This is necessary because the 
underlying causes of the age-performance curves often vary according to the 
domain of achievement. For example, the variables that account for the age 
curve in sports will not be the same as those that explain the curves in 
creativity, leadership, music performance, or chess. That restriction in mind, 


the historiometric work on the age—creativity relationship has arrived at the 


following four empirical generalizations (Simonton, 1988a, 1997, 2012; see 
also Jones et al., 2014; Kozbelt, 2014; McKay & Kaufman, 2014): 


1. The output of creative products in consecutive age periods is 
described by a curvilinear function. The output first rapidly increases to 
a single peak in the 30s or 40s and then gradually declines, producing an 
age decrement that approaches the zero output rate asymptotically. Most 
typically productivity in the final years of the career is about half the 
rate seen at the career peak. Hence, long-lived creators will remain 


productive. 


2. The specific location of the peak as well as the magnitude of the post- 
peak decline varies according to the particular domain of creative 
achievement. In some fields such as lyric poetry and mathematics the 
peak arrives relatively early and the age decrement is usually large, 
whereas in fields like history and geology the peak comes later and the 


ensuing decline is minimal. 


3. Properly speaking, the age functions just described are based on 
career age rather than chronological age. In any given field there will 
always appear considerable individual differences in the age at career 
onset (e.g. age at receiving a doctorate). Those with an accelerated onset 
(early bloomers) will have their career peak occur earlier in 
chronological age, whereas those with a delayed onset (late bloomers) 
will have their career peak appear later. The latter temporal shift is 


commonplace for those creators who exhibited a mid-life career change. 


4. The age-performance curves are largely the same for both quantity 


and quality of output. That is, the production of total works independent 


of creative impact follows approximately the same longitudinal form as 
the production of just those works that manage to exert some influence 
on the domain of creativity. As a consequence, a creator’s best work is 
more likely to appear in those periods in which the most total work 
appears. Contrary to what would be expected from a pure expertise 


perspective, a creator’s last work is seldom the best. 


It should be pointed out that many of the above results apply to other 
domains of outstanding achievement. For instance, the curvilinear function 
seen in creative domains has a very similar form in leadership, sports, and 
chess, where single-peaked functions are commonplace (Elo, 1965; Lehman, 
1953; Schulz & Curnow, 1988). At the same time, the specific form of the 
curve, including the location of the peak and the post-peak decline also varies 
from domain to domain. For example, the age for top performance in sports 
depends on the specific event (Lehman, 1953; Schulz & Curnow, 1988). On 
the other hand, some of the findings for creative achievement do not 
necessarily hold for other forms of world-class expert performance. A case in 
point regards the distinction between career and chronological age. To the 
extent that performance depends on physiological rather than psychological 
variables, the longitudinal curves will be defined in terms of chronological 
age. This qualification holds for some types of leadership and virtually all 


forms of sports (Schulz & Curnow, 1988; Simonton, 1998). 


Illustration 


A good example of historiometric research on expert performance is a study 


of top movie directors (Zickar & Slaughter, 1999). The specific goal was to 


determine the age-achievement function for 73 directors who made at least 20 
feature-length movies for the Hollywood film industry. Because the 
investigators used a hierarchical linear model, the units of analysis existed at 
two levels, directors and films. The films for each director were evaluated 
according to the ratings they received from film critics, as recorded in two 
movie guides. A composite evaluation constituted the dependent variable for 
the investigation. The independent variable was the order of the film in the 
director’s career. This order was introduced into the predictions equation in 
both linear and quadratic functions in order to test for the curvilinear single- 
peak function found in the literature on creative performance. In addition, 
acting quality was inserted as a control variable. This was gauged by the 
number of Academy Awards that each film received in the acting categories 
(two points for each lead role and one point for each supporting role). 

In general, the cinematic performance of the 73 directors was described 
by a quadratic function. That is, the ratings the films received from critics 
first increased to a single peak and thereafter declined. The most typical 
outcome was for a significant decline in performance to set in after the tenth 
film. Nevertheless, there were substantial differences across directors in the 
specific form of this curve. For instance, those directors who had higher rates 
of cinematic output tended to reach higher levels in peak performance, an 
effect found for virtually all creative domains (Simonton, 1997). More 


surprising was the finding that directors who launched their careers with an 


exceptionally successful film were most likely to exhibit a linear decline in 
performance rather than rise to a yet higher peak. This subgroup probably 
represents directors whose initial performance was due to luck, and thus the 


subsequent decrement can be attributed to regression toward the mean. 


Conclusions 


Although historiometrics cannot be considered a mainstream method in the 
behavioral sciences, the earliest research on expert performance used this 
technique. Furthermore, the number of studies that have accumulated since 
1835 is truly impressive (Simonton, 2014b). The resulting literature has 
produced an impressive body of empirical findings, particularly in those 
domains that entail creative expertise. Not only do we know a lot about the 
factors that underlie the acquisition of world-class expertise, but also we have 
learned even more about how that expertise manifests itself in adulthood 
performance. In fact, the cumulative results regarding the age-performance 
function in creative domains has become sufficiently rich and robust to 
provide the basis for complex mathematical models that can account for the 
fine structure of careers (Simonton, 1984b, 1989, 1997, 2010). Of course, 
there remain considerable gaps in our knowledge. This is particularly 
apparent in those domains outside creative achievement — most notably the 
diverse forms of leadership. In addition, even in domains pertaining to 
creativity we have much more to learn about the developmental correlates of 
expertise acquisition. Indeed, no matter what the domain, much more is 
known about expert performance than about how experts acquire the capacity 
to perform at world-class levels. 

Admittedly, historiometric methods have certain features that militate 
against its wide usage in empirical research. As a correlational method, it 
lacks the power of causal inference that is enjoyed by experimental methods. 


Historiometrics also has to rely on biographical and historical data that are 


sometimes of questionable reliability. Furthermore, historiometric inquiries 
focus on a subject pool or “research participants” that depart significantly 
from the norm. Rather than anonymous college undergraduates earning extra 
credit points in introductory psychology classes, historiometric samples 
invariably consist of individuals whose achievements have earned them a 
place in the annals of history (Simonton, 1999a, 2003b, 2014b). Samples of 
historic figures are not the first that come to mind when most researchers start 
to collect data! 

Nevertheless, the latter characteristic of historiometrics must also be 
viewed as one of its great assets. Any theory of expertise acquisition and 
expert performance must ultimately be able to account for those persons 
whose expertise reaches the highest possible levels. For example, a theory 
that explains how students solve science problems in laboratory experiments 
but not how real scientists earn Nobel Prizes must be considered woefully 
incomplete. Although other methods exist that permit the direct examination 
of eminent achievers, these have their own methodological limitations. For 
instance, Nobel laureates can certainly be subjected to intensive interview 
and assessment techniques (Feist, 2014; Nakamura & Fajans, 2014). Yet 
these methods depend on the willingness of such notables to participate. It 
should also be recognized that some of the liabilities of historiometric studies 
are being progressively removed. In the first place, the historical record is 
becoming far richer and more complete, in addition to becoming more 
available in electronic form on the Internet and computer storage media. Even 
more importantly, the statistical techniques suitable for the analysis of 
correlational data are becoming more sophisticated and powerful, thereby 
mitigating some of the problems in non-experimental causal inference. These 


Statistics include structural equation and hierarchical linear models as well as 


time-series analyses. By applying these tools to the lives and careers of the 
leading figures in the major achievement domains, it should be possible to 
enhance our scientific understanding of both expertise acquisition and expert 


performance. 
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Introduction 


The study of expertise in medicine has a long history, dating to early studies 
in radiology in the 1970s (Kundel & Nodine, 1975) and studies using general 
medicine cases and standardized patients (actors) (Barrows, Norman, 
Neufeld, & Feightner, 1981; Elstein, Shulman, & Sprafka, 1978). Medicine 
and surgery have some unique features that make the domain an attractive 
subject for psychologists. First, since all of us hope to maintain our health, 
effective healthcare delivery is a primary concern for virtually everyone, so 
understanding the nature of expertise and the source of errors may help to 
improve the health of the population. Second, unlike many of the domains of 
expertise described in other chapters, the multiple activities subsumed under 
the practice of medicine cover an enormous range of psychological 
mechanisms, from attention and perception to decision making to motor skill 
development. As a consequence, while deliberate practice has an essential 
role to play in the development of medical expertise, the nature and form of 
practice is likely very different for different domains. 

While the history of the study of medical expertise is long, it is not 
evident that this represents a sustained programmatic approach. Rather, while 
some areas such as radiologic diagnosis and basic skill development in 
surgery have attracted relatively sustained interest, others — patient 
management, clinical decision making, the role of emotions in reasoning — 
are underrepresented. Moreover, after a period of relative inactivity during 


the 1990s, there has been a recent burgeoning research interest in the study of 


medical expertise. This appears to be driven by two critical developments 


discussed below. 


The Institute of Medicine Report 


In 2000, the Institute of Medicine (IoM), a division of the National Academy 
of Science in the USA, published an influential monograph, To err is human 


(Kohn, Corrigan, & Donaldson, 2000), whose main thesis was that there was 


significant morbidity and mortality as a result of preventable medical error. 
Many of the errors highlighted were “system” errors — failure in 
communication, operation processes, and so forth. However one cause was 
diagnostic errors, which have been estimated to arise in 10-15 percent of all 


internal medicine cases (Graber, Franklin, & Gordon, 2005) and could 


potentially be ameliorated by a better understanding of the clinical reasoning 
process. This observation gained public and professional notoriety, leading to 
consensus conferences, new organizations (Society to Improve Diagnosis in 
Medicine), new journals (Diagnosis), and new standards in physician 
competency assessment (Frank & Danoff, 2007). These in tum led to a 
second IoM report, Improving diagnosis in medicine (Balogh, Miller, & Ball, 


2015), specifically directed at diagnostic errors. 


Simulation 


The past decade has seen exponential growth in the use of simulation 
technology to support teaching and assessment in medical education (Cook et 
al., 2011). This has taken various forms, from “virtual patients” (computer- 
based multimedia case presentations) to sophisticated robotic patient 
simulators costing tens of thousands of dollars to live actors trained to 
simulate a disease presentation (i.e. standardized patients) (Stillman et al., 
1986). The existence of these simulators has made it possible to create 
systematic approaches to learning for many aspects of clinical medicine, 
including symptom diagnosis, psychomotor skill acquisition from simple 
procedures like suturing to complex laparoscopic surgical procedures, and 
communication skills. 

These developments have fostered considerable growth in research 
directed at understanding the development of expertise in medicine. Some 
areas — such as the psychology of clinical diagnosis and the development of 
psychomotor skills — have seen considerable activity and feature prominently 
in this chapter. One developing area of interest, also highlighted, is the 
relation between emotions and reasoning. However, some areas, primarily 
related to patient management, have received relatively little attention as a 
research domain of interest. 

Finally, consistent with the general theme of the book, we explore 
research in medical expertise to examine how it may inform our 


understanding of the role of deliberate practice in expertise. 


Clinical Diagnostic Reasoning 


As described in the first edition of this handbook, initial research on clinical 
reasoning conducted in the 1970s revealed a common approach called the 


“hypothetico-deductive method” (Barrows et al., 1981; Elstein et al., 1978). 


Both experienced clinicians and medical students generated multiple 
diagnostic hypotheses within a few minutes of first contact with the patient. 
They then spent the remainder of the encounter gathering additional data to 
confirm or refute their hypotheses. A subsequent study in practice has shown 
that primary care physicians had 79 percent accuracy in diagnosis after just 
hearing the chief complaint (Gruppen, Wooliscroft, & Wolf, 1987). Perhaps 
even more impressive, radiologists are able to diagnose normal or abnormal 
with 70 percent accuracy after seeing a chest film for only 200 msec (Evans, 
Georgian-Smith, Tambouret, Birdwell, & Wolfe, 2013; Kundel & Nodine, 
1975). Thus, a key element of the reasoning process is the generation of 
hypotheses, very rapidly and effortlessly, very early in the patient encounter. 
However, a critical element emerged from these early studies: the 
finding of “content specificity” (Elstein et al., 1978). For both students and 
physicians, a good performance on one case was a poor predictor of 
performance on another case, even when content was quite similar (Norman, 
Tugwell, Feightner, Muzzin, & Jacoby, 1985), which is incompatible with the 
notion of a general problem-solving ability. Second, what distinguished 
clinicians from students was not the process per se; the most discriminating 
feature was simply that experts advanced more accurate hypotheses. The 


ubiquity of this finding led researchers to attempt to identify the kinds and 


organizations of knowledge that are critical in successful problem-solving. A 
number of possible knowledge structures were proposed including 


propositional networks (Patel & Groen, 1986), prototypes (Bordage & Zacks, 


1984), semantic axes (Bordage, 1994), and exemplars (Norman, Young, & 


Brooks, 2007). While each proposal found some support, the research was 
not sufficiently robust to definitely exclude one or another. However, in view 
of subsequent developments, a critical distinction does arise between the 
exemplar-based models that were linked with “non-analytic reasoning,” and 
the other forms which were viewed as variants on “analytical” reasoning 
(Norman et al., 2007). The distinction emerges more clearly below. 
Concurrent with these empirical studies, a second tradition emerged in 
the 1980s and 1990s, whose preoccupation was errors in the diagnostic 
process. Beginning with discussion papers by Elstein (1999) and others, the 
common theme was that diagnosticians were error-prone as a direct 
consequence of the limitations of human information processing. Drawing 


heavily on the seminal work of Tversky and Kahneman (e.g. 1974), the thesis 


is that the constraints of human memory force clinicians to rely on cognitive 


shortcuts or heuristics, which, while efficient, can lead to bias and subsequent 


errors. While dozens of heuristics have been described (Croskerry, 2003), 
very few have been verified experimentally (Lopes, 1991), and fewer still 
have been demonstrated in medical diagnosis (Eva & Norman, 2005; 
McLaughlin, Eva, & Norman, 2014; Zwaan et al., 2017). 

These two research streams have more recently merged into a “dual- 
process” model of reasoning (Croskerry, 2003; Evans, 2008; Evans & 


Stanovich, 2013; Kahneman, 2011). While there are many theories that vary 


in how they describe the two processes, the common element is that 


reasoning proceeds by two very different strategies: System 1 or Type 1 is 


rapid, automatic, and unconscious — the solution “pops into your head.” Type 
2 (System 2) is slow, analytical, and effortful. Type 1 reasoning in turn is 
based on cognitive shortcuts or heuristics that are highly efficient, but may be 
error-prone. Type 2, by contrast, is methodical and systematic, and loads on 
working memory. While the field has reached some degree of consensus 
regarding the nature of diagnostic reasoning, different terms are frequently 
used for the same or similar phenomena. The various descriptions are 
summarized in Table 19.1. Note that we have included “hypothetico- 
deductive” in this list; there appears to be a parallel between the original 


theory and more recent variants. 


Table 19.1 Labels and characteristics of Type 1 (System 1) and Type 2 


(System 2) thinking. 
TYPE 1 TYPE 2 
LABELS Hypothetico Deductive 
Non-Analytic Analytical 
System 1 System 2 
CHARACTERISTICS Fast Slow 
Unconscious Conscious 
Contextual General 


While dual-processing theory has been widely adopted as a theory of 


clinical reasoning (Croskerry, 2003), a number of points of contention 


remain. Fundamentally, the theory appears to describe both thinking 


Strategies (fast vs. slow, unconscious vs. conscious) and the kinds of 


memories retrieved. Logan and others (Logan, 1988; Smith & DeCoster, 


2000), for example, focus on Type 1 as a process of retrieving specific 
exemplars of solutions from memory. Conversely, the theory of Kahneman 
(2011) focuses on the kinds of shortcuts or “heuristics” used by Type 1 
processes and the potential for consequential errors. These heuristics are 
viewed as “hard-wired” and universal, with no particular relation to expertise. 
The majority of commentaries in medicine align with Kahneman’s (2011) 
view that errors arise primarily from biases in the cognitive heuristics 
associated with Type 1 processing. For example, Croskerry (2003, p. 32) 
stated: “Most errors occur with [System 1] and may to some extent be 
expected whereas [System] 2 errors are infrequent and unexpected.” 

In this view, the solution to error reduction is learn to recognize when a 
cognitive bias has arisen and then to use the analytical resources of Type 2 
processing to rationally review the process and correct the error. Ericsson 
(2004, 2011) has also implicated Type 1 processing as an impediment to the 


quest to become an elite performer: 


Consequently, the key challenge for aspiring elite performers in any 
domain of expertise is to avoid the arrested development associated with 
automaticity. Individuals striving for excellence need actively to 


counteract tendencies toward automaticity. 


(Ericsson, 2011, p. 7) 


The idea that errors arise solely from Type 1 is not universal, however. Evans 
and Stanovich (2013, p. 229) say: 


Perhaps the most persistent fallacy in the perception of dual-process 
theories is the idea that Type 1 processes (intuitive, heuristic) are 
responsible for all bad thinking and that Type 2 processes (reflective, 
analytic) necessarily lead to correct responses ... So ingrained is this 
good—bad thinking idea that some dual-process theories have built it into 


their core terminology. 


As one example of the latter, “premature closure,” which has been identified 
as the most common source of diagnostic error in medicine (Graber et al., 
2005), is noted by Evans and Stanovich (2013) as an example of the “sharply 
limited processing capacity” of working memory, which is used only by Type 
2 processes. 

A second debate is whether errors are a consequence of heuristics or 
knowledge deficits. If errors are caused by cognitive biases arising from the 
use of heuristics, remedial strategies should focus on learning to identify 
biases and applying strategies to overcome them. Conversely, if errors arise 
from knowledge deficits, strategies must enable marshaling additional 
knowledge resources. 

Recent evidence from studies in medicine has informed some of these 
issues and added to our understanding of the detailed nature of the dual 
processes. We will briefly address three assertions that arise from the 


theoretical positions stated above: 


(1) If errors are a consequence of cognitive biases, then we should be 
able to (a) show that these biases can be experimentally induced and (b) 


show that errors arising in practice are related to specific biases. 


Despite the fact that 30-40 cognitive biases have been described in the 


medical literature, relatively few have any empirical evidence in medicine. 


Perhaps the most studied is availability: the influence of a recent prior case 
on diagnosis of subsequent cases. A number of studies have examined this 
phenomenon, using a similar “two-step” approach, wherein participants first 
see a number of clear-cut cases under some pretext, then are shown some 
ambiguous cases, where the prior case can lead to a greater likelihood of the 
prior diagnosis being applied. Many of the early studies (Brooks, Norman, & 
Allen, 1991; Hatala, Norman, & Brooks, 1999) were used to verify an 
exemplar model of reasoning; more recent studies (Mamede, van Gog et al., 
2010) have used a similar design to establish availability bias. (The data are 
similar; the interpretation differs.) It should be noted that, while an exemplar 
based strategy is usually both effective and accurate, as demonstrated in some 
studies (Brooks et al., 1991), in most of these studies the prior exemplar is 
deliberately chosen to lead to error so it can be separated from other 
experiences. Consequently, in these construed circumstances, availability 
bias, related to Type 1 processes, does lead to errors, but the prevalence of 
the error in practice is unknown and likely unknowable. 

A few other biases have been studied experimentally. Weber and 
colleagues (Weber, Bockenholt, Hilton, & Wallace, 1993) examined cases 
that had both a common and rare possible diagnosis to look for base rate 
neglect; she found that experts were not vulnerable to this bias. Immunity 
from bias was also related to experience with similar cases. Christensen, 
Heckerling, Mackasey-Amitl, Bernstein, and Elstein (1995) looked at 
framing effects in prognostic decisions, and found little evidence of bias. 

It should be reiterated that these studies were designed to exemplify the 
role of cognitive bias in reasoning. As such, they cannot provide any insight 
into the actual prevalence of cognitive bias-related diagnostic errors. For that, 


one must look at actual performance. One study (Graber et al., 2005) 


examined 100 cases of diagnostic error and showed that multiple cognitive 
biases were present, most commonly “premature closure.” However, it is not 
clear whether this label reflects a stable bias or insufficient knowledge to 
obtain the critical information. Another study (Zwaan, Thijs, Wagner, van der 


Wal, & Timmermans, 2012) reviewed cases where errors were and were not 


committed and found that errors were primarily associated with knowledge 


deficits. 


(2) If errors arise in Type 1 heuristic processes and are resolved by 
application of rational Type 2 processes, then constraining the use of 
Type 2 processes by reducing available time or by adding distractions 


should increase errors. 


A number of studies have investigated the effect of time constraints and 
interruptions. In one observational study, Sherbino et al. (2012) showed that 
when participants were encouraged to go quickly, diagnostic accuracy was 
inversely related to time; more accurate diagnoses were arrived at more 
quickly. Similar results were obtained in an older observational study of 
dermatologists and other physicians and students (Norman, Rosenthal, 
Brooks, Allen, & Muzzin, 1989). 

An experimental approach, in which one group is encouraged to go as 
fast as they can and another is told to be slow, careful, and systematic, has 
been reported in several studies. Norman et al. (2014) showed a substantial 
difference in reading time, but no difference in accuracy between the two 
groups. Similar results were reported by Monteiro et al. (2015) and Ilgen et 
al. (2013). 

Interruptions can also inform process. Since Type 2 processing requires 


working memory, interruptions should reduce accuracy if Type 2 has an 


essential role in error reduction. One study (Monteiro et al., 2015) showed 
that two kinds of interruptions — an interposed multiple choice question and 
an auditory message — had no effect on accuracy. On the other hand, a recent 
study (Al Qahtani et al., 2016) did show that more severe time constraints 
may reduce accuracy. Explanation for the differences between studies 


remains speculative. 


(3) If errors are a consequence of cognitive biases, then error reduction 
can be achieved by teaching students to recognize biases in their 
reasoning. If errors are related to knowledge gaps, these interventions 
will be ineffective. However, there should be a correlation between 


accuracy and measures of knowledge. 


Several studies have attempted to teach cognitive biases and thereby 
reduce diagnostic errors. In one study (Reilly, Ogdie, Van Feldt, & Meyers, 
2013), students were better at identifying biases after instruction; in another 


(Bond et al., 2004) students perceived they had learned about biases. 


However, neither examined the effect on errors. Two studies that did look at 
errors with instruction on cognitive biases showed no effect (Sherbino, Dore, 


Siu, & Norman, 2011; Sherbino, Kulasegaram, Howey, & Norman, 2014). 


Conversely, a number of studies (Mamede et al., 2012; Mamede, Schmidt et 
al., 2010; Mamede, Schmidt, & Penaforte, 2008) have shown that an 


intervention directed at getting participants to mobilize diagnosis-related 


knowledge (“reflection”) has a small impact on reducing error. 


Summary 


The extensive research conducted in the past decade has reaffirmed and 
elaborated on the hypothetico-deductive model of reasoning first advanced in 
the late 1970s. Support for the model has emerged both from substantial dual- 
process theory building in psychology and also from considerable empirical 
research in medicine. While there has been much speculation about the nature 
of diagnostic errors based on the view that these arise from cognitive biases 
inherent in human reasoning, the evidence is more compatible with a 
knowledge-based theory of reasoning, where Type 1 processes reflect direct 
unconscious retrieval of prior exemplars (hypotheses) and Type 2 processes 
amount to extensive computational processing in memory directed at testing 
different rules. The evidence has also challenged the view that errors arise in 
Type 1 processes and are corrected by Type 2 processes. While the specific 
contribution of each remains to be elaborated, the evidence of a relation 
between errors and reliance on Type 2 processes has not been demonstrated. 
These findings have a direct bearing on our understanding of the nature 


of expertise in medicine. If errors resulted from hard-wired cognitive biases 


originating in the architecture of the mind (Kahneman, 2011), then an expert 
is presumably one who has successfully devised strategies to deal with the 
biases. Conversely, if, as the evidence suggests, expertise is a consequence of 
gradual accretion of both formal and experiential knowledge, then expertise 
is a consequence of mastery of a large and complex array of information. The 


two views arise again in our review of strategies to teach reasoning. 


Approaches to Teaching Clinical Diagnostic 
Reasoning 


Learning to diagnose clinical problems is a central objective of medical 
training. Historically, the acquisition of clinical reasoning has traditionally 
been left to the clinical years (the final years of medical school), where 
students encounter their first real patients and apply the knowledge acquired 
in previous years to diagnose their patients’ problems. These experiences are, 
however, characterized by learning-by-doing under irregular supervision and 
feedback, and the number and variety of patients to which students are 


exposed are not systematic (Rattner et al., 2001; Wimmers, Schmidt, & 


Splinter, 2006). In short, while there may be ample practice, there is rarely 


deliberate practice. These drawbacks have led many medical schools to 
develop curricula to teach clinical reasoning (Mandin, Harasym, Eagle, & 
Watanebe, 1995), but there is a paucity of empirical research on their 
effectiveness (Schmidt & Mamede, 2015). 


Research on how medical students’ diagnostic reasoning skills develop 


throughout the years of training may provide a basis for critically examining 
these approaches. Therefore, we first summarize this research before 
exploring the existing approaches. We then propose strategies to improve the 


teaching of clinical reasoning. 


How Clinical Reasoning Develops through Education 


The preceding discussion of dual-processing models advanced the theory that 
clinical reasoning involves two very different kinds of knowledge — 
experiential, “non-analytical,’ knowledge and conceptual, analytical 
knowledge. The evidence linking Type 1 processes to exemplars is quite 
strong (Norman et al., 2007). Conversely, there has been relatively little 
discussion in the literature about the kinds of analytical knowledge associated 
with Type 2 processing. However, a separate line of research focusing on the 
development of clinical expertise suggests that reasoning progresses through 
a number of transient stages, each characterized by knowledge structures that 
are qualitatively different from other stages (Schmidt, Norman, & Boshuizen, 
1990; Schmidt & Rikers, 2007). 

In the first years of training, students rapidly develop elaborated causal 
networks explaining the causes and consequences of diseases in terms of 
underlying pathophysiological processes. In this stage of development, 
students confronted with clinical problems do not yet recognize patterns of 
signs and symptoms that fit together. They tend therefore to focus on isolated 
clinical findings and engage in an effortful, detailed processing, trying to 
relate each finding to the pathophysiological processes they have learned. 

Evidence suggests that repeated application of such knowledge and 
particularly exposure to patient problems lead to a change in these knowledge 
structures. The pathophysiological knowledge of a disease becomes 
encapsulated into diagnostic labels or high-level, simplified models 


explaining signs and symptoms (Schmidt & Rikers, 2007). One consequence 


is that descriptions of pathophysiological mechanisms rarely arise in expert 


case discussion (Patel, Evans, & Groen, 1989) (although they do arise 
occasionally with complex cases; Norman, Trott, Brooks, & Smith, 1994). 
However, Woods, Brooks, and Norman (2007) found that understanding of 
mechanisms occupies a critical developmental role in learning reasoning. 
Those students who learned a causal explanation to relate a set of symptoms 
gave better diagnoses for similar cases with a different diagnosis presented to 
them one week later than students who just learned a diagnosis associated 
with these symptoms. 

As students begin to practice extensively with actual patients, a second 
shift occurs. They acquire more knowledge of the conditions under which 
disease emerges, the so-called enabling conditions (Feltovich & Barrows, 
1984), and begin to appreciate the variability of presentations of disease. 
Their encapsulated knowledge reorganizes into narrative structures labeled by 
Feltovich and Barrows (1984) and Schmidt and Rikers (2007) as illness 
scripts. These illness scripts are cognitive scenarios containing relatively little 
knowledge about pathophysiological causes of symptoms (because of 
encapsulation), but instead contain clinically relevant information about the 


variability in signs and symptoms associated with diseases. A final stage, 


according to Schmidt et al. (1990), is the acquisition of extensive exemplars 
of diseases, derived from actual clinical experience. Thus, to return to the 
dual-processing framework, the analytical knowledge of Type 2 consists of 
two very different knowledge representations involving (a) disease processes 
and (b) illness scripts. Type 1 processing relies on retrieval of specific 


exemplars of previously encountered patients. 


Curriculum Strategies to Enhance Clinical Reasoning 


Approaches to teaching clinical reasoning differ in several dimensions, but 
two are particularly relevant. First, the primary purpose of the educational 
intervention differs, reflecting an old distinction in the history of clinical 


reasoning research (Norman, 2005; Schmidt & Boshuizen, 1993) Some 


approaches aim at teaching students the experts’ reasoning processes as a 


general skill (Schmidt & Mamede, 2015). There is, however, consistent 


evidence, dating over four decades, that teaching strategies based on an 
assumption of a general skill are ineffective (Rogers, Swee, & Ullian, 1991; 


Round, 1999), which is not surprising considering the earlier discussion and 


other reviews highlighting the critical role of knowledge (Glaser, 1984; 
Perkins & Salomon, 1989). 

Based on this evidence, we will focus entirely on curriculum strategies 
that reflect the acquisition of the different kinds of knowledge described in 
the previous section. A common goal of these formal curricula is to create the 
opportunity for practice with many, and many different, clinical problems, 
arranged in a systematic fashion, while ensuring appropriate supervision and 
feedback; in short, to create a forum for deliberate practice within the 
undergraduate medical experience. While the research is not extensive, two 
ideas are fundamental. First, teaching should focus on acquisition and 
application of knowledge, rather than a general process of reasoning. As we 
discussed earlier, the notion of a general reasoning process lacks theoretical 
and empirical support. Second, as described previously, expertise in 


diagnosis involves multiple kinds of knowledge. As expertise progresses 


through transient stages, an approach should foster the shifts in knowledge 
structure expected to happen over different phases of training. 

Specifically, as we described earlier, knowledge proceeds from an 
acquisition of causal mechanisms, to an encapsulation phase, then to 
acquisition of clinical knowledge and then exemplars. Initial teaching 
approaches should enable students to explicitly practice explaining signs and 
symptoms in terms of their pathophysiological sources, which encourages 
students to understand how human biology produces disease and creates 
coherence among seemingly disparate signs and symptoms. 

To help students develop adequate scripts for particular diseases, 
teaching approaches must be employed that enable them to compare and 
contrast similar-appearing diseases. Finally, in the final years of 
undergraduate training and beyond, the emphasis is on acquiring multiple 
exemplars of diagnoses. While historically, these exemplars were acquired 
haphazardly in the course of clinical rotations, the opportunity arises to 
dissociate clinical education, to some degree, from the clinical setting, and 
thereby devise evidence-based instructional approaches to _ teaching 
reasoning. Strategies of deliberate practice and interleaved practice may 
result in considerably more effective learning, although currently evidence is 


lacking. 


Specific Instructional Strategies 


Various instructional strategies differ in fundamental ways. Whereas some 
approaches present all information about the clinical problem at once (the 
“whole-case” approach), others present case information to students in a 
sequential format (the “serial-cue” approach). The latter simulates a real 
clinical encounter: students “play doctor” and are requested to gather, step- 
by-step, the information necessary to arrive at a diagnosis (Goss, 1996; 
Kassirer, 2010). 

The serial-cue approach appears to be the most prevalent (Mamede & 
Schmidt, 2015). As it appears to more closely simulate a real clinical 
encounter, teachers may think that it would better prepare students for 
practice. However, there is little evidence of effectiveness (Littlefield, Hahn, 
& Meyer, 1999; Windish, Price, Clever, Magaziner, & Thomas, 2005). 


Moreover, when studies compared the two approaches, students’ diagnostic 


accuracy was significantly lower in the serial-cue than in the whole-case 
format (Nendaz, Raetzo, Junod, & Vu, 2000). 

By contrast, there is some experimental evidence of the effectiveness of 
whole-case approaches oriented toward acquisition or application of 
knowledge that focus either on developing knowledge of the underlying 
mechanisms of diseases or on increasing students’ ability to distinguish 
between diseases that share similar clinical presentations (Mamede & 
Schmidt, 2015). 

A similar concern with making the simulation as accurate as possible 
has led to the development of simulations using videos and standard actor 


patients. However, again, the evidence suggests that lower fidelity text-based 


simulations provide an equivalent opportunity for learning and transfer 
(Durning et al., 2012; La Rochelle et al., 2011). 


There are various effective strategies to encourage mastery of relevant 


knowledge. One strategy to enhance learning of causal knowledge involves 
self-explanation; while diagnosing clinical cases, students read the case and 
explain to themselves how the findings in the case could have been produced 
by underlying pathophysiological mechanisms, how these signs and 
symptoms relate to each other and to the possible diagnoses for the case. One 


study (Chamberland et al., 2011) had students initially diagnose clinical cases 


either with or without self-explanation. In an assessment phase, one week 
later, students from the self-explanation group performed better. Subsequent 
studies confirmed the benefits of self-explanation (Chamberland et al., 2015). 

An alternative approach is to simply read a basic science explanation 
linking signs and symptoms to diseases. In a series of studies, Woods 
(Woods, Brooks & Norman, 2005; Woods et al., 2007) showed that students 


who read a brief basic science explanation performed significantly better than 


a comparable group who simply memorized the features of various diseases. 
These findings suggest that understanding the mechanisms of a patient’s 
symptoms helps students to better understand how these symptoms are linked 
together, apparently providing coherence to a mental representation of the 
disease and making it easier to recognize subsequently. 

A third knowledge-oriented approach focuses on knowledge of the 
clinical presentation of diseases rather than underlying mechanisms. Briefly, 
the approach requires students to practice with clinical cases that portray 
similar patient presentations by comparing and contrasting the clinical 
findings of alternative diagnoses for each case. As such, this is a variant of 


the “mixed practice” strategy (Taylor & Rohrer, 2010). An example may 


illustrate the approach. Two experiments tested an instructional procedure 
that encourages “deliberate reflection” while diagnosing internal-medicine 
cases (Mamede et al., 2012, 2014). After providing an initial diagnosis for a 
case, students were asked to identify findings in the case that either 
corroborate or oppose this first diagnosis, generate alternative diagnoses for 
the case, and proceed with the same analysis for all diagnostic hypotheses, 
until a most likely diagnosis is reached. In a first experiment, fourth-year 
medical students who practiced with clinical cases by using the deliberate 
reflection procedure outperformed students who employed free reflection 
when diagnosing new cases of the same diseases one week later (Mamede et 
al., 2012) A follow-up study found deliberate reflection to lead to higher 
performance also on cases that were similar to the cases studied but in fact 
had a different diagnosis (Mamede et al., 2014). 

Other studies have also supported the value of approaches based on 
contrasting alternative diagnoses for teaching clinical reasoning (Lee et al., 
2010) and ECG interpretation (Hatala, Brooks, & Norman, 2003). Comparing 
and contrasting alternative diagnoses while reflecting upon a case seems to 
enrich mental representations and influence the representations of related but 
different diseases, making it easier to discriminate among confusable 


diagnoses. 


Summary 


While we are achieving a reasonable understanding of the process of clinical 
reasoning, these theories are only beginning to be translated into effective 
instructional strategies. The approaches reviewed are not strongly linked to 
aspects of the reasoning process, and appear to be somewhat “ad hoc.” 
However, underlying the various strategies are two common elements — 
elaboration, and mixed practice — that have some basis in our understanding 
of the nature of learning. As the healthcare system evolves to optimize 
delivery at the possible expense of education, the opportunity arises to 
dissociate clinical education, to some degree, from its historical basis in the 
clinical setting, and thereby devise evidence-based instructional approaches 


to teaching reasoning. 


Expertise in Surgery and the Development of 
Psychomotor Skills 


While the discussion so far has focused largely on expertise in terms of 
diagnostic ability, a significant component of medical practice also involves 
technical skill. In the earlier edition of this handbook (2006), we reported that 
there had been very little work on expertise in surgery and related technical 
fields. This is no longer the case. There is a great deal of ongoing research 
relating surgeon volume and years of experience to patient outcome and 
many studies on the training and assessment of technical competence in 
health professional, medical, and surgical training. As we pointed out at the 
beginning of the chapter, one major impetus for renewed interest is the 
publication of several high-profile works such as To err is human (Kohn et 


al., 2000). A second force for change is a consequence of changes in the 


healthcare system, in both North America and Europe, such as legislation 
setting limits on working hours and the increasing management of patients 
outside of the traditional hospital in-patient environment (Woodrow, Seguin, 


Armbruster, Hamstra & Hodges, 2006). The focus of the field has been on 


ensuring the development of surgeons with the requisite breadth and skill to 
maintain public trust in surgical education given these new constraints 
(Berwick, 2015b; Nasca, Day, & Amis, 2010; Ulmer, Wolman, & Johns. 


2008). Finally, increasing use of simulations, running the gamut from partial 


task trainers to sophisticated virtual reality, has led to increasing research 


related to learning technical skills. 


Objective Measurement of Technical Skills 


The execution of clinical procedures using motor skills involves more than 
simply movement proficiency. Certainly, success in a comprehensive surgical 
operation requires a deep knowledge of functional anatomy, managing the 
flow of the operation, intra-operative decision making, and inter-professional 
communication skills involving a diverse team (Anastakis et al., 1999; 


Wanzel, Hamstra, Anastakis, Matsumoto, & Cusimano, 2002). Nevertheless, 


most patients (and surgeons) would emphasize operative or manual skill in 
any discussion of expertise in surgery. The nature of these skills permits 
opportunities for the collection and analysis of performance metrics related to 
quality of precision movements, and in tum provision of feedback on 
progression toward technical expertise. 

Several approaches have been used. The Objective Standardized 
Assessment of Technical Skills (OSATS) (Reznick, Regehr, MacRae, Martin, 
& McCullough, 1997) and its derivatives have been widely adopted in 


surgical education. OSATS is based on the Objective Structured Clinical 
Examination (OSCE) (Harden & Gleeson, 1979), a widely used approach to 


assessing clinical skills. OSATS, like OSCE, consists of a number of brief 
stations in which the candidate is asked to perform a surgical procedure under 
close observation The key to its success is that it highlights the specificity of 
skills, and creates a total score by averaging over a number of individual 
dimensions of performance, including respect for tissue, instrument handling, 
use of assistants, and procedural flow. A recent review by Ghaderi et al. 
(2015) used this framework to match a suite of assessment instruments to a 


standardized national curriculum in surgery endorsed by leading national 


organizations in that field. They were able to classify a total of 23 assessment 
instruments for the 35 curriculum modules for technical skills. 

Beyond subjective assessments, technical skills often have outcomes 
that are amenable to objective measurement. For example, Frischknecht and 
colleagues (2013) developed a process for objective assessments of suture 
closures on an artificial skin pad. Their analysis identified a number of suture 
outcomes that were related to surgical expertise, including total bite size, 
stitch orientation, and the ratio of symmetry across the incision. 

Furthermore, data about movement efficiency and kinetics has been 
used to differentiate experts and novices. The Imperial College Surgical 
Assessment Device (ICSAD) uses an electromagnetic motion tracking system 
to provide information about surgical efficiency, with the implication that 
expert surgical skill is characterized by shorter distances traveled by the 
surgeon’s hands and a fewer number of total movements made during an 
operation (Datta, Chang, Mackay, & Darzi, 2002). Rojas et al.’s (2010) 
development of a custom suturing simulator revealed that expertise in 
surgical knot-tying can be determined by measures of knot tightness and 
stability as well as movement economy. Movement proficiency-associated 
measures of technical skill expertise have also revealed experienced 
acupuncturists to have greater and more consistent velocity and amplitude 
control during needling (Li, Grierson, Wu, Breuer, & Carnahan, 2013). 

Overall, each of these metrics — whether presented to the learner in an 
augmented fashion or perceived by the learner intrinsically with performance 


— can be used as feedback to drive deliberate practice. 


Simulation 


Many of these metrics are necessarily generated and used in the context of 
simulation-based education. In a recent meta-analysis, Cook and colleagues 


(Cook et al., 2011) showed that simulation-based clinical education was 


associated with learning benefits for medical trainees. Central to this learning 
benefit is the idea that the low-stakes and patient-free nature of simulation- 
based learning permits learners to take advantage of a number of the features 
of deliberate practice that are not permissible during real-life clinical 
procedures, such as clear task-related goals, repetition, and augmented, real- 
time feedback (Issenberg, McGaghie, Petrusa, Gordon, & Scalese, 2005; 
McGaghie, Issenberg, Cohen, Barsuk, & Wayne, 2011; Tsuda, Scott, Doyle, 
& Jones, 2009). 


However, because simulated technical procedures are inherently 


different than technical procedures performed in the clinical setting, it is 
important to consider the validity of assessment measures, feedback 
information, and learning that results in this context. When simulators were 
first introduced, they were characterized with respect to their technological 
sophistication and visual similarity to reality; often called fidelity (Allen, 
Buffardi, & Hays, 1991). Over time, fidelity was refined into several 
domains. Engineering fidelity reflected the physical similarity between the 
simulated and actual contexts. Environmental fidelity described the degree to 
which the simulation mirrored the sensory stimuli associated with true 
performance. Psycho-affective fidelity referred to the degree that performers 
perceive the simulation to be a surrogate for the real clinical task (Issenberg 
& Scalese, 2008; Rehmann, Mitman, & Reynolds, 1995), 


However, recent reviews of the relationship between the degree of a 
simulator’s defined fidelity and learning reveal no effect of fidelity (Norman, 
Dore, & Grierson, 2012; Zendejas, Brydges, Hamstra, & Cook, 2013). 
Indeed, several studies have shown that technical skills acquired on low- 
fidelity bench models transfer equally as well as high-fidelity models to 
improved performance on criterion models (such as human cadavers) and live 
patients (Anastakis et al., 1999; Matsumoto, Hamstra, Radomski, & 


Cusimano, 2002; Grober et al., 2004). An explanation is that transfer occurs 


because the low-fidelity models preserve functional aspects of the 
psychomotor skills that are important to the criterion surgical skill, a concept 
recently termed “functional task alignment” (Hamstra, Brydges, Hatala, 
Zendajas, & Cook, 2014). That is, the definition of fidelity based on 
technological sophistication does not reflect the true nature of psychomotor 
skill learning and may be misleading in the context of specific learning 


objectives. 


Simulation, practice, and mastery learning 


Because the simulation-based learning environment affords trainees the 
opportunity to engage in deliberate practice of technical skills, it has become 
an important context for the application of mastery learning approaches to the 
education of health professions. Mastery learning is a form of competency- 
based education in which learning objectives for a particular skill are 
translated into a series of fixed standards of progressive difficulty. It requires 
that the learner undergoes baseline testing, engages in deliberate practice, and 
receives rigorous formative assessments with respect to the predetermined 


achievement standard for each learning objective. The student is not 


permitted to move on to the next objective until she or he meets the minimum 


performance standard for the previous objective (Carroll, 1963; McGaghie, 


2015; McGaghie et al., 2011). In this way, mastery learning is purported to 
result in the efficient achievement of skills with little or no variation between 
learners. 

In the health professions, mastery learning has been reported to result in 
superior skill development, but can take more time than alternative 
approaches (Cook et al., 2013). Evidence for the success of mastery-based 
training approaches has been shown for the skills of a number of medico- 
surgical techniques including those relevant to laparoscopic surgery 
(Zendejas et al., 2013), lumbar puncture (Barsuk et al., 2012; Kessler, 
Auerbach, Pusic, Tunic, & Foltin, 2011), and catheter insertion (Barsuk, 
Cohen, McGaghie, & Wayne, 2010). 

The positive impact of mastery approaches on technical skill education 
in surgery and medicine has been shown across a number of relevant 
measures of mastery, including performance measures taken in the simulation 
laboratory (Wayne, Barsuk, O’ Leary, Fudala, & McGaghie, 2008; Barsuk et 


al., 2012; Zendejas et al., 2011) and in patient care contexts (Kessler et al., 


2011; Gauger et al., 2010). Importantly, a few studies have even shown 
positive impacts of mastery learning in patient outcomes, which is an area of 
study fraught with complexity. For instance, mastery learning conducted in 
the simulation environment is associated with improved laparoscopic 
inguinal hernia repair surgery (Zendejas et al., 2011) and reduced 
bloodstream infections associated with catheter insertions (Barsuk, Cohen, 
Feinglass, McGaghie, & Wayne, 2009). 


Surgical Expertise and Specificity of Practice 


A growing body of research shows that technical expertise in surgery is 
acquired, and highly local. The ability to perform one task derives from 


specific practice with that task and does not generalize to other, even 


apparently similar surgical tasks (Wanzel et al., 2002). At the heart of this 
interpretation is empirical evidence that supports theories of motor control 
that rely on the operation of cortical representations of actions (e.g. Schmidt, 
1975). These representations develop with experience and define the 
sensorimotor processes that the performer relies on to ensure that skilled 
movements are completed accurately. 

Particularly relevant in this regard is Starkes and colleagues’ (Starkes, 


Payk, Jennen, & Leclair, 1993) seminal work on microsurgery proficiency 


and training. Microsurgery is different from traditional open surgery in that 
the operations are guided visually by use of a microscope. At the time of 
Starkes et al.’s research, these techniques were only just becoming adopted in 
North American hospitals and presented a unique opportunity to explore the 
degree to which procedural content knowledge (i.e. knowing) and motor 
proficiency (i.e. doing) interact to characterize surgical expertise. To this end, 
a novice surgical trainee and an experienced open surgeon were invited to 
practice sutures in the microsurgery environment for the first time while their 
performances were measured. Interestingly, the first performances in the new 
environment demonstrated that the experienced open surgeon was at a 
disadvantage despite a wealth of experience with, and declarative knowledge 
about, the suturing procedure. Starkes et al.’s work highlighted the contextual 


expertise that is defined by the specific features that surround the acquisition 


of the skills. That is, the expert surgeon’s suturing proficiency was not 
transferable to the microsurgery environment because it was developed with 
particular respect for aspects of the open surgery environment. 

The issue arises in the use of simulators. Van Sickle et al. (2007) found 
that experienced laparoscopic surgeons’ performance on a series of simulated 
laparoscopic tasks presented on a virtual reality simulator was not correlated 
with their years of practice or the number of laparoscopic procedures 
completed, but was rather with the surgeon’s accumulated experience on the 
specific simulator. Similarly, Park et al. (2007) found that simulator-based 
metrics of performance were ineffective predictors of performance in the 
clinical setting, leading to questions regarding effective simulator design. 

This is the idea of specificity of practice that is critical in defining best 
practice in the development of surgical expertise (Elliott, Grierson, Hayes, & 
Lyons, 2011) and is central to definitions of simulation fidelity (Grierson, 
Lyons, & Dobrowski, 2013; Hamstra et al., 2014; Norman et al., 2012). It has 
parallels to the earlier discussion of content specificity and the role of 
experience in clinical reasoning. 

The fundamental assumption is that skills must be performed in dynamic 
and unpredictable contexts that have not been previously experienced. The 
representational perspective of skill performance (Elliott et al., 2011) does 
not explain how skills are performed in new contexts or with greater levels of 
complexity than previously encountered. When experts perform well in these 
situations, there must be processes that supersede memory-based 
applications, because memory-based transfer is limited (Starkes et al., 1993). 

One alternative perspective on human motor performance (which we 
call “constraints-based”) accepts that there are a number of different ways 


that a task may be performed, and that the decision to perform any one of the 


options is constrained by the interactions that exist between the practitioner, 
the procedure, and the clinical context (Davids, Button, & Bennett, 2008; 
Newell, 1989). In this view, it is through an accumulation of experience of 
completing a certain task amid various constraints that permits the 
development of skills that can be adapted effectively to new contexts 


(Berstein, 1996). Interestingly, both the representational (Elliott et al., 2011) 


and the constraints-based (Davids et al., 2008; Newell, 1989) perspectives 
recognize that progressive, deliberate practice that increases the complexity 
of a task — for example, through variable outcome goals, spaced repetition, or 
feedback delays — improves performance at retention. But the latter 
acknowledges a strategy for generalizing to new contexts. 

For example, Grierson et al. (2013) compared the degree to which 
novices learned basic laparoscopy skills when they practiced on a simple 
monitor arrangement compared to a more complex, visually rotated monitor 
arrangement. Learners were able to improve to equivalent levels under both 
conditions; however, when their acquired skill was tested under the opposite 
monitor condition, the group that practiced on the complex monitor had little 
success transferring to the simpler monitor arrangement, while those that 
practiced in the simple monitor condition outperformed the complex 
condition learners on the complex arrangement. The interpretation is that 
simpler conditions of practice permit learners to acquire key, functional 


aspects of a task in a more transferable manner. 


Summary 


Research on the development of surgical expertise is complementary to the 
literature on the development of clinical reasoning, in that the emphasis is on 
motor skill development. While there is some divergence, such as a more 
circumscribed role for deliberate practice, aligned with the literature on 
expertise in athletics, music, and other motor-based skills, nevertheless 
common threads, such as specificity of practice/content specificity, emerge 


from both traditions. 


Predictors of Medical and Surgical Expertise 


The premise of the theoretical framework of deliberate practice (Ericsson, 
2004, 2011; see also Ericsson, Chapter 38, this volume) is that with sufficient 
time and supervised training by a teacher anyone can dramatically improve 
their performance with very few known immutable limits imposed by genetic 
endowments (e.g. height and body size). One demonstration of the central 
role of deliberate practice is to identify individuals at various levels of 
expertise, as measured by an acknowledged standard, and then examine the 
growth of their expertise as a function of time devoted to practice. What 


emerges is a now classic learning curve that only asymptotes after thousands 


of hours of practice (Ericsson, 2004), for example, the gradual increase in 
EILOS rating of chess players as a function of hours of practice (Charness, 
Tuffiash, Krampe, Reingold, & Vasyukova, 2005), and the similar growth 
curves associated with learning musical instruments (Ericsson, Krampe, & 
Tesch-R6émer, 1993), although Mosing, Madison, Pedersen, Kuja-Halkola, 
and Ullén (2014) have proposed exceptions (but see Ericsson, 2016, and 
Chapter 38, this volume for a different interpretation). It would appear self- 
evident that a similar curve would arise in medical expertise, and some 
research has shown similar learning curves in very specific domains such as 
learning to read an ankle X-ray (Pusic, Boutis, Hatala, & Cook, 2015; Pusic, 
Pecaric, & Boutis, 2011) and laparoscopic cholecystectomy surgery (Crochet 
et al., 2011) although the time dimension is typically replaced by numbers of 
repetitions, and may correspond to only a few hours of training. However, 


comparable curves do not exist in broader areas of competence, likely 


because of the highly variable nature of patient presentations in most medical 
and surgical practice. It is also clear in studies in areas as diverse as chess and 
athletics that improved performance is restricted to the domain of practice, 
which we have labeled “content specificity” in the medical domain and 
“specificity of practice” in the surgical domain. An important corollary is that 
performance in clinical practice is unrelated to measures of aptitude such as 
IQ or memory capacity (Ericsson & Charness, 1994). 

So why should we examine predictors of expertise? One reason is that 
medicine does have some unique affordances. In contrast to many other areas 
of expertise, pursuing a career in medicine results in documented 
achievement on a large number of objective measures, from those recorded 
prior to admission — undergraduate grade point average and medical college 
admissions test scores — to performance in national licensing examinations 
and speciality certification examinations offered by every specialty board in 
the USA and Canada, which may provide insight into individual differences 


that may be predictive of expertise. 


Experience, Expertise, and Elite Performance 


In many of the domains studied in this book, the focus of study has been the 
true elite — the top 1 percent of practitioners — grand master chess players or 
their equivalent. Few studies in medicine have attempted to identify the very 
best; indeed, because of the diversity of skills involved in medicine it is not 
clear what defines the “best.” One area that has been studied is diagnostic 
performance in internal medicine, using peer nomination. One of the earliest 


studies of clinical reasoning (Elstein et al., 1978) studied peer nominated 


physicians but was unable to differentiate them from a comparable group of 
physicians that were not nominated. More recently, Eva et al. (2011) used an 
elaborate survey method to identify high-performing physicians at five 
academic centers. When exceptional physicians were interviewed about their 
success, they identified four common themes — continued deliberate learning 
from their practice, skills to organize and classify their patient “stories,” 
reflection on their activities, and continuous practice to promote enhanced 
performance. However, the study did not identify whether they were, in fact, 


superior diagnosticians. 


Predictors of Expertise in Medicine 


Age and expertise in medicine 


Since expertise in many domains is so strongly dependent on an 
accumulation of thousands of hours of deliberate practice, it is natural to 
suppose that experience and expertise are highly related, so that measures like 
years in practice, which reflect an accumulation of patient experiences, may 
be a surrogate for expertise. This is generally not the case (Ericsson, 2004). A 
number of reviews have examined the relationship between age and various 


measures of clinical performance (Durning et al., 2010; Ericsson, 2004; Eva, 


2002). All have shown the same finding; a small but consistent negative 
relationship with age (Ericsson, 2004). The finding may reflect the distinction 
between practice and deliberate practice, the latter involving systematic 
exposure with graded difficulty and feedback. 

It may also be the case that age is a poor surrogate for direct experience. 
Norcini, Kimball, and Lipner (2000, 2002) found that a measure of practice 
activity — number of patients cared for — was negatively related to mortality 
in the coronary care unit (i.e. positively associated with expertise), while time 


since graduation was related to increased mortality. 


Formal knowledge and practice measures 


As we indicated earlier, in contrast to many other domains, the study of 
medicine is accompanied by performance on many achievement and aptitude 
tests, some of which are available for ongoing research. This permits an 


examination of the relation between knowledge and performance. As 


Ericsson (2004, p. S79) says, “If we recognize that medical performance 
likely depends on pre-existing knowledge and skills, we must consider the 
possible role of individual differences in previously acquired skills.” 

Studies of national licensing examinations typically show large 
correlations between earlier tests of knowledge and performance tests (Case 
& Swanson, 1993). Ramsey et al. (1989) showed a 0.6 correlation between 
performance on a written specialty examination and practice performance 
assessed by chart review ten years later. More recently, Tamblyn et al. (2007) 
showed that a written multiple choice licensing examination was a better 
predictor of complaints to a regulatory body than measures derived from a 
performance examination (an OSCE) as part of the same examination. 
Norcini et al. (2002) looked at the relation to a patient outcome measure — 30- 
day mortality in the coronary care unit following a heart attack. Passing the 
written specialty examination resulted in a 19 percent lower mortality. More 
recently Norcini, Boulet, Opalek, and Dauphinee (2013) studied performance 
of international medical graduates (IMGs) on a written licensing examination 
(USMLE 2 — Clinical Knowledge). There was a negative relationship 
between examination score and mortality; a one-standard deviation change in 
score was equivalent to a 4 percent absolute change in mortality. 

Thus, while gross measures of experience such as years in practice or 
age (essentially the same thing) show weak or negative relationship to patient 
outcomes, measures related to individual aptitude do show a consistent 


positive relationship to outcome. 


Predictors of Expertise in Surgery 


Age, time, and expertise in surgery 


In contrast to medicine, in surgery, there is an accumulating body of evidence 
that increasing professional experience with a specific procedure is associated 
with better outcomes for patients. Prominent in this regard is the work of 
Vickers and colleagues (Vickers et al., 2007, 2008, 2009). In this series of 
studies, the researchers examined the way that surgical mortality rates change 
as a function of the number of procedures completed, and found that 
prostatectomies completed by practitioners with experience of less than ten 
such procedures were almost twice as likely to result in a recurrence of the 
cancer than those completed by surgeons with experience of more than 250 
procedures (Vickers et al., 2007), that cancer recurrence in a specific organ 
was reduced over a surgeon’s first 1,500 to 2,000 procedures (Vickers et al., 
2008), and that this type of experience-based recurrence reduction was 
demonstrable for practitioners of both laparoscopic and open surgical 
procedures (Vickers et al., 2009). Atug et al. (2006) found that tissue 
removed during radical prostatectomy surgery was less likely to contain 
cancer around the sample’s edges when the surgeon had more experience at 
the operation. Kelles, Barreto, and Guerra’s (2009) study of bariatric surgery 
revealed that the mortality rate 30 days after bariatric surgery was 
considerably higher for patients of surgeons that performed fewer surgeries 
per year. Cho, Yu, and Kyung (2010) showed a similar relationship between 
the number of root contacts during dental surgery and the surgeon’s level of 
experience. Prystowsky (2005) found a positive association with years of 


experience and outcomes of complex surgery. Finally, Bilimoria et al. (2009) 


found in two of four studies that increasing time since graduation was 
associated with better surgical outcomes; two found no association. 

In an intriguing study, Norcini et al. (2013) found, in looking at cardiac 
surgery, that with procedures in which an attending physician was present, a 
positive outcome was associated with less years since graduation of the 
surgeon, but more years since graduation of the attending physician (the skill 
of the young; the wisdom of the old). 

Overall, the data suggest that much of the variability in technical 
procedure outcomes is directly related to the experience of the surgeon in 
performing the specific procedure. One possibility is that the relationship 
between expertise and number of procedures performed in surgical 
operations, but not in other aspects of medicine, is a function of the nature of 
technical tasks and the immediate, regular feedback about the success or 
failure of a procedure (Ericsson, 2011). With technical procedures, errors and 
successes are often apparent immediately during an operation or very shortly 
thereafter. This information is a critical component of deliberate practice. 
However, it is less common that a final, discrete diagnosis is available to a 


physician in a way that helps the clinician improve their diagnostic ability. 


Summary 


The picture emerging from these studies is that there is an inconsistent 
relation between very gross measures of practice experience such as age or 
years since graduation and performance. But this weak association masks a 
more complex picture. Direct measures of experience — specific patient 
experience, overall patient volume — show a consistent positive relationship 
with performance. Further, there emerges a consistent positive relationship 
between aptitude measures assessing formal knowledge and performance, 
even over years and decades. In surgery there is a more direct relationship 
between specific surgical experience and patient outcomes, which we suggest 
is a consequence of the ease of provision of timely feedback, a precondition 


for deliberate practice. 


Future Issues 


The studies we reviewed present an incomplete picture of expertise in 
medicine. We have focused exclusively on two areas — diagnosis and 
technical psychomotor skills. A number of other areas were omitted simply 


because there is insufficient evidence to draw broad conclusions. 


Management Decision Making 


Research into management decision making has taken a very different course 
than diagnosis, resembling more the “heuristics and biases” literature in its 
emphasis on suboptimal decision making. Evidence-based medicine (Guyatt 
et al., 1992), whose premise is that clinical decisions should be informed 
primarily by evidence from clinical research (usually randomized trials), has 
resulted in a proliferation of “practice guidelines” that prescribe the “correct” 
approach to various clinical issues. In turn, departures from guidelines are 
viewed as indicative of suboptimal practice (Choudhry, Fletcher, & 


Soumerai, 2005). However little is known about how or why these departures 


arise. One recent study (Mercuri et al., 2015) suggests that these represent 


rational decisions based on consideration of the whole patient and unique 


environmental contexts, not simply the medical dimensions of the problem. 


Role of Emotion in Reasoning 


When considering the causes of diagnostic errors, it is natural to implicate 
non-cognitive factors like stress, which may in tum affect thinking. 
Remarkably, there is little evidence to substantiate these conjectures. Four 
studies (Harvey, Bandiera, Nathens, & LeBlanc, 2012; Harvey, Nathens, 
Bandiera, & LeBlanc, 2010; LeBlanc et al., 2012; Pottier et al., 2013) have 


experimentally manipulated the stressfulness of a simulated medical 


situation. In all studies, the manipulation resulted in higher cortisol levels 
indicative of a stress response; however, only two studies showed any effect 
on diagnostic accuracy. In a different research program, Mamede et al. (2017) 
showed that a written patient case that included a description of an angry 


patient was sufficient to reduce accuracy. 


Situational and Structural Effects 


Many healthcare settings are chaotic; it is natural to presume that the 
necessity to multitask or time pressures may cause errors. Such is not 
necessarily the case; some studies have shown minimal impact of both 
interruptions and time pressure on accuracy (Ilgen, 2013; Monteiro et al., 
2015). A recent study (Al Qahtani et al., 2016) did show an effect of severe 


time constraints, which may have been sufficient to induce anxiety. 


Environmental factors may result in communication breakdowns during 
“handover” when healthcare teams change and patient information must be 
communicated. While each of these factors has some supporting literature, 


much more research is required. 


Conclusions 


The topics covered are far-ranging, reflecting the diverse dimensions of 
medical expertise. While the complexity of the domain precludes simple and 
universal conclusions, it does provide a basis for a richer understanding of the 
nature of medical expertise. Three themes have emerged from this review that 
exemplify the unique characteristics and affordances of the domain: (1) the 
integration of analytical and experiential knowledge that contributes to many 
aspects of clinical competence, from technical psychomotor skills to 
reasoning and decision making, (2) the contribution of individual attributes 
and aptitudes to expertise, and (3) as a consequence, the complex relationship 


between experience and expertise. 
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Introduction 


Forging valid empirical generalizations about expertise in the transportation 
domain is a daunting task given the large number of operators, different types 
of air, land, and sea vehicles, and vast infrastructures. To understand the 
relationship between expertise and transportation, the broad domain of 
transportation can be broken down into the different modes of transport. At 
the most basic level, the modes of transport are air, land, water, pipeline, and 
space. Land transport can be divided into road and rail. As our discussion will 
show, different modes of transport have led to different forms of expertise. In 
air transport, expertise is most regularly investigated in the forms of flying 
fixed-wing aircraft, flying rotorcraft, and controlling air traffic. In land 
transport, expertise is most regularly investigated in the form of driving cars, 
followed by riding motorcycles, and operating trains. In sea transport, 
expertise is investigated in the form of maritime driving, although less 


frequently. Expertise in space and pipeline transport is not often investigated. 


People-Moving across Our Planet 


Transportation, people moving from one point on this planet to another, is a 
domain that likely boasts more experts than any other. The large number of 
operators alone justifies continuing research on experience in transportation. 
The value of experience in transportation has been documented for a number 
of years, reflected in everyday life in accident records, insurance rates, and 
legislation. Research in transportation dates back to the early twentieth 
century, with selection and aptitude tests of military (Henmon, 1919) and 
transport pilots (Snow, 1926). Research on experience and driving can be 
traced back at least to the work of Allgaier (1939), who showed that although 
reaction time and visual acuity were best for the younger driver, accident 
rates and traffic violations decreased with experience. 

Studying expertise in transportation also pushes our understanding of 
expertise generally by extending it to dynamic environments. Many studies of 
expertise take place in static or less than fully dynamic environments. 
However, in transportation, the environment will change whether or not the 


operator takes an action. Expertise in modern dynamic environments (see 


Cellier, Eyrolle, & Marine, 1997, for a review) has been of interest since the 
1970s with studies of process control (DeKeyser & Piette, 1970), aviation 
(Bisseret, 1971), and driving (Mourant & Rockwell, 1972). 


Moving out of the lab and away from static tasks toward dynamic ones 


lets us test our explanations of expertise in more complex environments. For 
example, while it may not be surprising in chess to find expertise effects, 


showing that cognition is also important to expertise in driving or flying 


expands our understanding. Indeed, transportation offers complex 
embeddings of cognitive and non-cognitive factors in an operator who is 
using technology that is embedded in a dynamic environment that in turn is 
embedded in a social setting of a rich culture. This adds complexity to our 
theorizing about expertise. 

Complexity carries a price. Overall, studying outcome variables “in the 
wild,” such as safety in transportation, will entail multiple causes, only one of 
which will be experience. For example, drivers differ in their styles and 
willingness to accept risk. They are also able to engage in compensatory 
behaviors, allowing more distance by driving farther back when they believe 
their speed of reaction is slowed (e.g. Groeger, 2000). Age is an obvious 
entanglement with experience, be it through the cognitive decline that comes 
in the later years or the variations in risk tolerance that accompany 
adolescence and young adulthood. Thus, we might expect the relationship 
between experience and representative performance to be weak (see Ericsson, 
2006) when we leave the comfortable environs of more controlled studies. 

In automobile driving, the poorest drivers are the very young — those 


who have recently received a license (Malone & Brunken, 2013) and the very 


old (>80), with the groups having comparable accident rates (OECD, 2001). 
Licensed drivers who have three to five years of experience, and are typically 
in their 20s, have a much lower rate of accidents, similar to drivers in their 
60s and early 70s who may have close to 50 years of driving experience. 
Although cognitive decline is operating in these 60- and 70-year-olds, their 
performance in the transportation domain does not seem to be comparably 
impaired. 

In 1973, Quenalt and Parker showed experience effects when age was 


controlled. Since then it has become clear that expertise can often compensate 


for cognitive decline (e.g. Taylor, Kennedy, Noda, & Yesavage, 2007). 


Operators can learn to self-regulate their behavior by using compensatory 
strategies (e.g. Motak, Gabaude, Bougeant, & Huet, 2014; Yeung & Wong, 
2015). Reflected in accidents, insurance rates, and scientific studies, 
experience affects driving performance and safety even when colliding 
against aging. 

Legislation also recognizes the value of experience. As in many 
industries, experience in transportation is a critical determinant of 
advancement. Graduated driver’s licenses are required in many states in 
America. The more time an apprentice drives, the more responsibility the 
driver is permitted (e.g. night driving privileges, more than one passenger). 
As a driver’s proficiency level progresses through the apprentice and 
journeyman stages, commercial driving privileges become attainable. 
Likewise, in aviation, a trainee pilot is permitted to fly solo, but not allowed 
to take passengers until obtaining at least a private pilot’s license (limited to 
one passenger for a recreational pilot’s license). A minimum of 1,000 hours 
(and in many cases 750 flight hours) is required as a first officer in an 
airliner, and most airlines will only promote to captain status pilots who have 


several thousand hours. 


Transportation in the First Edition of the 
Cambridge Handbook 


In the first edition of this handbook, Durso and Dattel (2006) reviewed the 
literature on expertise and transportation by attempting to classify the results 
according to underlying cognitive mechanisms. Durso and Dattel used 
empirical results and discussions in the literature to induce research 
categories of Attention (including dual-tasks and visual search), Perception 
(including hazard detection and_ situation awareness/understanding), 
Knowledge (including organization and mental/situation models), and 
Strategies. 

Many studies suggested that expertise is not only processing information 
automatically, but also involves superior management of resources. Even 
when one might normally expect automatic processing to be critically 
important, evidence of deliberate management of resources was apparent in 
experts. The resources managed could be available time to perform a task, 
time sharing, workload, and so on. Resource management deficits seemed to 
characterize non-experts. Superior resource management could then vie with 
the notion of automaticity to explain findings, such as that experts are less 
affected by load. In fact, Durso and Dattel suggested that some aspects of the 
transportation task (e.g. vehicle control) can become automatic because of the 
consistent mapping between relevant stimuli and responses, whereas other 
aspects of the transportation task (e.g. hazard detection) cannot because 


“there is simply not enough consistent mapping between some contextualized 


hazards and the appropriate response” (Durso & Dattel, 2006, p. 363). 


Superior resource management was not the only characteristic of experts 
in transportation that emerged from the earlier edition of this handbook. 
Many studies suggested that expert knowledge is crucial. Compared to non- 
experts, experts identified situations faster, better discriminated relevant from 
irrelevant, handled discrepancies between expectations and reality more 
efficiently, applied declarative knowledge to real-time situations well, and 
were better at predicting the results of (some) of their actions. The strategies 
experts used seemed more flexible than those of novices, even though they 
tended to use fewer strategies. To Durso and Dattel (2006), in dynamic 
worlds like transportation, rather than increases in facts, expertise brings 
improvements in strategies. 

Much of the material in Durso and Dattel’s (2006) chapter in the first 
edition of this handbook is still valid, and so we refer interested readers to 
that chapter. In this chapter, we articulate some empirical generalizations 
about experience and transportation based on work conducted since 2006. 
The majority of empirical articles reviewed here are about driving. At least 
part of this preponderance reflects a focus since 2006 on automobile driving 
in the literature. And rather than organizing the literature in terms of 
underlying cognitive mechanisms, we base this chapter’s organization on 
generalizations about key findings. Obviously, inducing generalizations is an 
uncertain process, but in our opinion, they represent the culmination of most 
of the methodologically sound work in this area. These empirical 


generalizations can also serve as heuristics to generate future empirical work. 


Obtaining Expertise Effects Requires the 
Capture of the Domain of Expertise 


The relationship between experience and a variable such as safety raises the 


question: What kind of experience (McKinney & Davis, 2003)? What 


constitutes expertise varies by transportation domain. Expertise in 
commercial flying is a function of the aircraft; an expert flying a regional jet 
may not be an expert flying a 757. On the other hand, train engineers 
specialize on specific geographic areas rather than type of train. En route air 
traffic controllers develop areas of specialization, contiguous sectors of 
airspace. 

Although it is true that experience can improve performance, more 
experienced participants do not outperform less experienced ones in every 
empirical study. A good example of the complexity that can arise in the wild 
from differing experiences and confounding variables is Williams and 
O’Neill’s (1974) finding with licensed racecar drivers of Sports Car Club of 
America. It turns out that their racecar drivers had more on-the-road accidents 
than did controls. We might consider that Williams and O’Neill’s drivers 
may be more risk tolerant; or cars chosen by drivers with a fondness for 
racing may differ from other cars. Certainly, the type of experience gained 
during a race may not be the kind of experience that aids safety on public 
roads. So, for several reasons the hoped-for reduction in accidents in these 
“expert drivers” did not materialize and the notion of a “special” Masters 


Driver’s License in the United States gained no traction. 


Failure to find expertise effects is not always due to gross mismatches 
between experience and the performance tasks being assessed. Part of the 
complexity evident in the literature can be due to subtle differences in 
methods. For example, failures to obtain a hazard perception advantage as a 
function of experience can, at least sometimes, be traced to not effectively 
capturing the target domain in which the participants are presumed to differ 
in experience. A common case is the inclusion of items not relevant to, for 
example, driving. Expertise effects might only manifest for some of the items 
in such a study. 

An illustration of how tenuous expertise effects can be in transportation 
comes from Zhao et al. (2014). They explored the conflicting literature that 
has accumulated regarding driver experience in change detection. They 
hypothesized that when participants were not allocating attention as they 
normally would during driving, an expertise effect would not appear. Zhao et 
al. compared those with 33 months of driving experience with non-drivers 
either when they were made to fixate the scene as they would when driving, 
or when they were free to allocate attention as they chose. Experienced 
drivers outperformed non-drivers, but only when allocating attention as they 
would be required to do while driving. With the freedom to scan as they 
pleased, non-drivers allocated attention in a way that was adaptive for the 
change detection task. Ultimately Zhao et al. concluded that “drivers’ 
increased ability to detect changes will only appear when the task requires a 
pattern of visual attention distribution typical of actual driving” (p. 998). The 
Zhao et al. study is evidence that the expertise advantage in transportation 
may require capturing specific, sometimes non-obvious, features of the target 


domain. Otherwise, the expertise effect can be quite ephemeral when tested 


in situations that are merely analogous to the domain but not reflective of the 


expert’s familiar task. 


Experts Exhibit More Flexible Behaviors Than 
Do Novices 


In our review of the literature, it seemed to us that when the transportation 
situation changed, experienced drivers and experienced pilots were more 
likely to change their behavior to adapt to the new situation. Less experienced 
operators may show little change because of the situation change. For 
example, for less experienced drivers (under two years of driving), a quiet 
rural road or a busy multi-lane road made little difference in visual 


information processing (Crundall & Underwood, 1998). 


Indeed, many studies show that experts are more varied than novices 
when the situation changes. For example, Mueller and Trick (2012) asked 
experienced (junior journeyman/expert averaging eight years of driving) and 
less experienced (apprentice with an average of less than half a year) drivers 
(all under 35) to drive in clear and fog conditions. In clear conditions, the 
more experienced drivers drove faster than the less experienced drivers. Of 
relevance here is that the experienced group reduced their speed significantly 
more because of the fog than did the less experienced group. 

In another example, Godwin et al. (2013) varied sea severity as non- 
drivers (no prior maritime driving experience) and experienced drivers (mean 
16 years driving; minimum 4 years) controlled a powerboat simulator. 
Experienced drivers’ fixation durations decreased and their horizontal spread 
increased as the seas became rougher, showing good adaptation to the 
features of the waves relevant to negotiating rougher waters. Non-drivers, on 


the other hand, were more variable in the wrong direction (vertical) as 


attention was captured by salient but less relevant “surface” structure. 
Godwin et al. summarized that “inexperienced drivers show inflexibility in 
adaptation to changing driving conditions” (p. 27). 

Similar results were found by Crundall, Stedmon, Crundall, and 
Saikayasit (2014), who were interested in why motorcyclists who account for 
1 percent of the vehicle miles traveled are involved in 18.5 percent of the 
fatalities. Overall, experienced riders (three years riding) tended to vary their 
lane position more than less experienced riders (completed compulsory basic 
training), who tended to ride more closely to the curb. This difference in 
experience disappeared if the road had a medium or tight curve. Thus, 
experienced riders engaged in more flexible positioning unless the road 
geometry was difficult and presumably resource demanding (see Rogers, 
Kadar, & Costall, 2005, discussed below). 


In a study looking at the black hole illusion in aviation (when there is no 


ground reference, such as flying at night over water), Gibb, Schvaneveldt, 
and Gray (2008) were surprised to find that less experienced pilots (average 
250 hours) landing in impoverished conditions flew more stable approaches 
than experienced pilots (average 6,000 hours). Gibb et al. argued that the 
experienced pilots were altering their pitch in an effort to seek and better 
understand the visual cues available in that perceptually uncertain situation. 
The pitch changes due to such a proactive search process on the part of the 
experienced pilots would account for their less stable slope. 

In both Crundall et al. (2014) and Gibb et al. (2008), the more 
experienced driver or pilot actively modified the situation by changing either 
lane positions or aircraft pitch. Thus, not only are experienced operators more 


likely to respond to a situation change with varied behavior, they are more 


likely to initiate varied behaviors to better adapt to the continual changes of a 
dynamic environment. 

So, experts are more likely to change behaviors to adapt to a new 
situation. But what is a new situation? This has been a question dodged by 
most human factors research and those who study situation awareness. 
Clearly being in a dense fog versus clear atmosphere are different situations, 
as are rough seas and calm seas. However, what about being in a wispy fog? 
Or moderate seas? Might such a situation be functionally the same as a clear 
day? 

Maybe the answer is yes for experts and no for novices. Returning to 
Godwin et al. (2013), both experienced and non-drivers slowed their boats 
going from moderate to rough seas. However, it was non-drivers, not experts, 
who slowed their boat going from slight to moderate seas. Godwin et al. 
(2013) argue that the difference in the slight and moderate seas were 
inconsequential for experts, resulting in experts driving more quickly through 
moderate waters than could naivettes. The reader is invited to consider the 
Cognitive Flexibility Theory of Paul Feltovich and his colleagues (Feltovich, 


Spiro, & Coulson, 1997), in regard to our first empirical generalization. 


Gaze is Less Constrained with Expertise, 
Unless There is a Threat 


Of course, both experienced and inexperienced operators show context- 
dependent variability in their behaviors. Of course, both experienced and 
inexperienced operators show context-dependent variability in their 
behaviors. Gaza in driving is one behavior that has been extensively 
researched. In driving, gaze reflects two primary intentions: to direct attention 
toward the surfaces over which drivers intend to travel, and to direct attention 
toward surfaces which drivers intend to avoid traveling over (Rogers et al., 
2005). Numerous studies have compared gaze patterns of experienced and 
less-experienced operators in efforts to understand performance differences 
and improve training. Some of these studies found expertise effects and 
others did not, sometimes in the same domain. Sometimes novices will even 
show more variance than experts on a particular variable: the glances of 
apprentice (400 to 15,000 km) drivers vary more than those of experts 
(50,000 to 2,000,000 km), and comprise proportionately more extreme short 
(ineffective) or long (dangerous) glances from the road to a shared task 
(Wikman, Nieminen, & Summala, 1998). 


More recent work points to evidence that task relevant variability often 


increases with experience. Most studies have found that gaze becomes more 
variable in the horizontal plane with experience. For example, in a straight 
road driving simulator task, unlicensed participants with no driving 
experience had gaze distributions that were closely centered to the middle of 


the road ahead; more experienced apprentices (<10 hours high-speed driving) 


had gaze distributions that were farther out to the left and right sides of the 
road; and even more experienced advanced apprentices/junior journeymen 
(>100 hours high-speed driving) had gaze distributions that included the 
periphery and side of the road (Rogers et al., 2005). The same effect was 
found in a curved road driving task in which advanced apprentices/junior 
journeymen (<5,000 km driving) spent more time gazing at the road ahead 
than journeyman/expert drivers (30,000 km) who spent less time gazing 
ahead and more time fixating over curves (Lehtonen, Lappi, Koirikivi, & 
Summala, 2014). 

The effect obtains in higher workload tasks as well. For example, a 
study of driver behavior at railroad crossings found that advanced 
apprentices/junior journeymen (provisional license; 3 years driving) spent 
more time glancing at the forward roadway whereas journeymen/experts (full 
license; 17 years driving) were more broadly focused (Young, Lenné, 
Beanland, Salmon, & Stanton, 2015). A simulator study that included day, 
night, and rain driving also found that apprentices (licensed; 24 hours 
driving) restricted their fixation to the scene directly in front of them, 
whereas experts/exemplary journeymen (average 34 years of driving 
experience) had a greater number of shorter fixations distributed more widely 
across the driving scene (Konstantopoulos, Chapman, & Crundall, 2010). 

In a high-speed scanning task, advanced apprentices/junior journeymen 
(<3 years driving) were not sensitive to changes to road complexity, whereas 
journeymen/experts (>20 years driving) adapted more extensive gaze patterns 
on roads that demanded increased monitoring. Experts/senior journeyman, in 
this case trained police drivers, adapted their gaze patterns even more 


(Underwood, 2007). Similarly, in the maritime piloting task of Godwin et al. 


(2013) participants with no maritime driving experience were inflexible in 


adapting their gaze to changing sea conditions whereas journeymen/experts 
increased the horizontal spread and decreased the duration of fixations in 
rough seas. The effect was also found in an overland navigation task where 
novices and advanced apprentices/junior journeymen did not change their 
scan patterns when navigation difficulty increased but journeymen/experts 
increased their scanning time and decreased dwell time (Yang, Kennedy, 
Sullivan, & Fricker, 2013). 

The effect can be found in complex tasks as well. For example in high- 
speed, low-altitude, helicopter flight, advanced apprentices spend more time 
looking out the window, whereas journeymen had more fixations on the 
instrument panel (Kirby, Kennedy, & Yang, 2014). In VFR (visual flight 
rules) landings, novices had longer dwell times and consecutive fixations out 
the window whereas journeymen/experts had more total fixations that 
alternated between the window and the altimeter (Kasarkis, Stehwien, 
Hickox, & Aretz, 2001). A model of the influence of expertise (ranging from 
novice to expert/senior journeyman) on visual scan patterns in helicopter 
overland navigation also found that for every additional 1,000 helicopter 
flight hours of expertise the number of view changes increases by 33 and 
median dwell time decreases by 28 seconds (Sullivan, Yang, Day, & 
Kennedy, 2011). 

Taken together, these findings demonstrate that gaze is flexible with 
expertise in simple, high workload, variable, and complex driving tasks in 
cars, ships, planes, and helicopters. However, it is important to note that all of 
the tasks in these studies found that gaze was less constrained in the direction 
of surfaces drivers intend to travel over or through. In cars, gaze was less 
constrained to the road ahead so drivers could do trajectory planning over 


curves. In planes and helicopters, gaze was less constrained out the window 


ahead so pilots could monitor other instruments. In ships, gaze was less 
constrained to the ocean ahead so drivers could monitor the horizon. 

Studies that have investigated the effect of expertise on tasks that 
involve surfaces that drivers intend to avoid, or in other words threats, have 
found other results. For example, the same study that found a difference 
among naivettes, apprentices, and advanced apprentices/junior journeyman in 
a straight road driving task found no such difference in a driver braking task: 
gaze was always highly constrained toward the intended braking path and 
anchored on the intended stopping point, regardless of expertise (Rogers et 
al., 2005). The same was true for a steering into lane task where drivers’ gaze 
patterns did not vary as a function of expertise (Kadar, Rogers, & Costall, 
2011). If a train was present at railroad crossings both less experienced and 
more experienced drivers took more frequent and longer glances in the 


direction of travel (Young et al., 2015). 


Experts Anticipate 


Experience leads to the development of better strategies and better mental 
models, which can be used to direct attention or anticipate the future. Our 
review of attention allocation above tells us that experienced drivers differ 
from novices in where they look, and why they look where they look. We 
have known for some time that less experienced apprentices concentrate their 
search in a smaller area, closer to the front of the vehicle, whereas 
journeymen look at the focus of expansion (e.g. Mourant & Rockwell, 1972). 
This difference in gaze can be used to help experts anticipate. More recent 
work elaborates on the relationship between anticipation and experience, and 
points to differences in perceptual memories. 

Experts tend to anticipate more than novices (Blattler, Ferrari, 


Didierjean, & Marmeche, 2012; Blattler, Ferrari, Didierjean, van Elslande, & 


Marmeche, 2010). For example, when videos of road scenes resumed at the 
exact place that they stopped, expert drivers (~2 hours/day for 18 years) 
remembered the stopping point as being more forward than did novices (no 
license; Blattler et al., 2010). This anticipatory bias is modulated by specific 
knowledge of the moving object within a domain (Didierjean, Ferrari, & 
Blattler, 2014) and is not transferred to dissimilar domains (Blattler et al., 
2010). The same effect was found in aviation, where expert pilots (~3,000 
hours) had more anticipation than participants with no experience, but only in 


the direction of travel (Blattler, Ferrari, Didierjean, & Marmeche, 2011). 


Hazard Perception, and Not Vehicle Control 
Skills, Predicts Crash Risk and On-Road 
Driving Performance 


Horswill, McKenna, and colleagues have advanced this proposition for years. 


(See Gugerty, 2011, for a review of situation awareness models and measures 


in driving.) Numerous studies have shown a relationship between experience 
and hazard perception, with experts responding faster in the lab (McKenna & 
Crick, 1991) and better on the test track (e.g. Lee et al., 2009). Their 
performance is superior in both daytime and nighttime conditions (Garay- 


Vega, Fisher, & Pollatsek, 2007). These results usually manifest even though 


more (>8,000 km for at least ten years) and less experienced drivers (less 


than four years) can be shown to rate situations similarly when not under time 


pressure (e.g. Wallis & Horswill, 2007). Indeed, in our first review, we were 
quite convinced that expert drivers would outperform novices in hazard 
perception. The empirical work since our last review has painted a more 
complex picture. 

Hazard perception — detecting and interpreting an event as a risk to safe 
driving — has been the focus of both recent empirical work to understand 
driving safety and even recent legal changes in driver licensing. In fact, 
Horswill, Hill, and Wetton (2015) were the first to demonstrate a connection 
between a government-issued hazard perception test (Queensland, Australia) 
and subsequent crashes. 

Hazard perception is usually thought to be a non-automatic cognitive 


process (e.g. McKenna & Farrand, 1999) with a substantial top-down 


component. The fact that hazard perception requires resources means that as 
drivers master vehicle control skills, ultimately to achieve automaticity, 
mental resources are freed up and can be devoted to noticing more of the 


environment (Patten, Kircher, Ostlund, Nilsson, & Svenson, 2006), including 


perceiving hazards (Durso & Dattel, 2006). The fact that there is a top-down 
component implies that as drivers gain more knowledge and develop mental 
models to organize perceptions, then hazard perception should improve. 

Laboratory studies of hazard perception typically have the participant sit 
in front of a monitor and press a button when the situation presents a danger 
(see Jackson, Chapman, & Crundall, 2009, for a presentation of the task’s 
evolution). By not requiring vehicle control, such studies disentangle those 
processes from hazard perception. 

There are many aspects of the driving situation that lead to the 
expectation that experience will improve situation awareness and hazard 
perception. As the driver experiences an increasing sample of driving 
situations, knowledge of the road — its statistical relationships between cues 
and probability of danger — increases. Thus, whereas novices might respond 
based on individual factors like risk tolerance and impulsivity (e.g. Xu, Li, & 
Jiang, 2014), more experienced drivers should be more likely to respond 
based on the cue—hazard contingencies on the road. 

However, there are also many aspects of the driving situation that leads 
one to question whether or not mere experience will necessarily improve 
hazard perception. First, hazards are relatively infrequent. In addition, they 
are usually avoided, and if they go unnoticed, that would reduce the number 
of opportunities for feedback still further. In addition, the feedback itself is 
usually vague and uncertain, leading to a weak connection between a cue and 


a hazard. Much of today’s research focuses on the cue—hazard relation. 


Experienced Drivers Are Superior at (Some 
Types of) Hazard Perception 


One feature of the more complex picture painted in today’s literature is that 
some cue—hazard contingencies show an effect of experience and others do 
not. Novices (1.5 months of licensed driving experience) reacted more slowly 
than experienced drivers when a pedestrian was still obscured (Borkowsky & 


Oron-Gilad, 2013), but when the pedestrian crossing the street was visible, 


novices tended to be more comparable to journeyman drivers (mean = 7.6 
years) and expert (23.5 years) taxi drivers. Work by Crundall et al. (2012) 
suggests that this may be more than simply a distinction between perception 
and interpretation. 

Crundall et al. (2012) analyzed eye fixations in situations when a hazard 
was signaled by the presence of a precursor. In some cases, the precursor was 
directly connected to the hazard, for example a car waiting to merge 
(precursor) and then actually pulling in front of the driver (hazard). In other 
cases, the hazard was foreshadowed less directly. For example, there might 
be a parked truck with flashing hazard lights on the side of the street 
(precursor) and a delivery man carrying boxes steps out from behind the 
truck (hazard). (Crundall et al. refer to the direct and indirect scenarios as 
behavioral and environmental hazard scenarios, respectively.) Apprentices 
(7.5 months of experience) missed more direct precursors than did the 
experienced (16.4 years post-license) and instructor drivers (30 years), but 
detected the same number of direct hazards. On the other hand, indirect 


precursors were fixated equivalently across the three levels of expertise, but 


the hazards themselves were missed more often by the apprentices. So, 
everyone fixated on the car pulling in front (hazard) and the parked truck 
with flashers (indirect precursor), but experience increased fixations on the 
car about to merge (direct precursor) and the delivery man stepping in front 
of the car (hazard). 

Why does experience matter for the delivery man stepping in front of 
our car but not for the car pulling in front of us? In all of Crundall et al.’s 
direct scenarios, the precursor became the hazard: the merging car actually 
merges; the child seen between two parked cars steps into the road; the 
motorcycle signaling to turn, turns in front of the car. In the indirect 
scenarios, the hazard was physically distinct from the precursor: the ice 
cream truck presages the child, the parked truck presages the delivery man, 
and the blind bend presages a broken down car. It may be the case that 
experience allows us to connect otherwise superficially different objects and 
therefore help us develop complex causal connections. This may be a general 
principle in transportation. For example, in aviation Schriver, Morrow, 
Wickens, and Talleur (2008) found that whereas both apprentice (110 hours) 
and junior journeymen (482 hours) benefited from diagnostic cues, only the 
more experienced group made fast decisions when the cues were correlated 
(see also Scialfa et al., 2012). 

We know that novices organize the perceived world according to 
“surface similarities,’ and when those surface similarities lead to predicting 
the future as was the case with direct scenarios, then novices should do as 
well as experts. Borkowsky, Oron-Gilad, and Parmet (2009) provided some 
indication that experienced drivers tend to classify movies of driving by 
considering both the similarity of hazards as well as the similarity of the 


traffic environment; novices only classify based on hazard similarity. 


Thus, superior knowledge allows experts to deal with more complex 
situations, such as indirect cue—hazard connections and correlated cues. 
When that knowledge is made less useful, as with random chess positions, 
expertise effects are diminished or eliminated (e.g. Chase & Simon, 1973). In 
transportation, when experienced drivers are deprived of the ability to use 
their knowledge, experience effects have been reversed. Borkowsky, Shinar, 
and Parmet (2008) presented novices (six months’ driving experience) and 
journeymen (five years’ driving experience) with 160 photographs of traffic 
scenes, each followed by a yes/no question. Eight critical stimuli asked about 
the presence of a No Left (Right) Turn sign that appeared in expected 
locations or unexpected ones. The usual expertise effect manifested only 
when locational information was consistent with expectation. In fact, novices 
outperformed experts when location information was violated. Borkowsky et 
al. used eye fixation data to confirm that the effect was due to experts 
expecting signs to be on the right, whereas novices did not seem to favor a 


location. 


Minutes of Good Training on Hazard 
Detection Can Be Equivalent to Years of 
Driving Experience 


We concluded in our earlier review for this handbook that hazard perception 
should be a skill that could benefit from training: “Explicit training on pattern 
detection and interpretation of hazards, the development of strategies, and the 
management of resources are likely targets for ... deliberate practice” (Durso 


& Dattel, 2006, p. 369). Unlike most traditional driver training interventions 


which fail to improve safety (Beanland, Goode, Salmon, & Lenne, 2013), 
there is evidence that training hazard perception can have practically 
significant effects. In a nice review of the literature, Horswill (2016) makes 
an argument for using hazard perception training, suggesting that even brief 
interventions can be valuable. There is indeed reason for optimism. Thomas, 
Blomberg, Peck, and Korbelak (2016) used a 17 minute training procedure 
from Pradhan, Pollatsek, Knodler, and Fisher (2009). In some instances the 
results were impressive: a year later, (male) trainees had a crash rate 24 
percent lower than their untrained counterparts. 

Again using a short video, Wallis and Horswill (2007) compared 
advanced apprentices (2 years) who received hazard perception training or 
not with untrained experts (30 years). The training engaged the participants 
by asking them to “... think about sources for potential hazards by asking 
themselves what might be reasonably expected to happen” (p. 1181). 
Training made apprentices more like experts: quick to respond to hazards and 


biased to respond to more situations as potentially hazardous. The short 


engagement training made participants more like those with years of “mere 
experience.” 

“Mere experience” is thought to be insufficient for the development of 
expertise and expert performance (Ericsson, Krampe, & Tesch-Rémer, 1993). 
In hazard perception, one can easily see why mere experience is a poor way 
of acquiring the skill. First, the frequency with which hazards occur is low. 
Second, feedback is poor. Third, the events experienced naturally are not 
selected or structured with pedagogy in mind. Fourth, the drivers are not 
trying to put themselves in situations where they may fail. However, all of 
these can be designed into a training program. 

Nevertheless, merely addressing these issues — say by having students 
watch video clips of hazards and pressing a button when a hazard is detected 
— is not typically effective. In fact, Horswill (2016) notes that his team uses 
such interventions as the placebo control condition to evaluate experimental 
interventions. However, it does not necessarily take much beyond this 
placebo condition to help drivers improve. Hazard perception seems to be 
improved when the participant is engaged, by having to identify the hazard; 
when the participant listens to an expert describe what they were looking at 
and what they were anticipating; or when students narrate out loud followed 
by listening to an expert narrate the same film (see Horswill, 2016, for a 
description of a number of training interventions). 

“Deliberate practice” is often juxtaposed to mere experience. Although 
deliberate practice in all its detail has not been implemented into a driver 
training or a hazard perception training program, the more successful training 
programs contain components consistent with deliberate practice as opposed 


to ineffective training which resembles “mere experience.” 


Meta-Processing Varies with Experience 


Some of the differences observed in decision making between the 
experienced and less experienced can be attributed to many of the processes 
discussed thus far. These processes also give rise to differences in meta- 
knowledge of one’s skill and abilities. Cestac, Paran, and Delhomme (2011) 
found that more experienced drivers (three years or greater) would only 
exceed the speed limit if they felt that they had good control of the vehicle. In 
addition, speeding for experienced drivers was a more conscious intent than it 
was for less experienced drivers. 

These experienced drivers reported risks (e.g. the probability of a 
speeding ticket) as influential factors in their intent to speed. Less 
experienced drivers’ main reason for speeding was their previous behavior; 
that is, the more likely a less experienced driver reported speeding in the past, 
the more likely he or she would speed in the future. Therefore, the more 
experienced drivers surveyed in Cestat et al.’s study (also see Motak et al., 
2014) are more likely to self-assess their capability and external factors (e.g. 
speeding ticket) as factors in their decision to speed. 

This meta-knowledge utilized by more experienced operators seems to 
carry over to team dynamics. Meade, Nokes, and Morrow (2009) tested naive 
pilots (no aviation experience), apprentice pilots (student pilots who did not 
have a private pilot’s rating), and expert pilots (flight instructors who had a 
commercial pilot’s license with an instrument rating) on their recall after 
reading aviation scenarios. After reading simple and complex scenarios, 


participants wrote down as much about the scenario as they could recall. 


Participants wrote down information individually, or with another participant 
of the same expertise level (collaborative group). In the collaborative group, 
one of the two individuals was instructed to write down the information. The 
dyad was free to communicate to each other what each recalled. 

Expert dyads recalled more information than apprentice dyads who 
recalled more information than naive, non-pilot dyads. Moreover, apprentice 
dyads and naivette dyads recalled less information than they did as 
individuals. 

There were also differences in behavior. For example, experts were 
more likely to read back information and restate previous information. 
Because most pilot operations consist of other team members (even solo 
pilots are likely to communicate with air traffic control), awareness of each 
other’s primary goals and competencies emerges. With experience, pilots 
develop shared mental models with their team members. Consequently, 
experienced pilots know how to utilize team members’ qualities and 
characteristics to maximize performance. 

Of course, a final meta-judgment that is critical to transportation is 
making judgments of risk. Less experienced drivers not only respond more 
slowly to driving hazards as we reported earlier (Scialfa et al., 2012; 
Stankovic, Raufaste, & Averty, 2008), but they also report that driving 


hazards are less of a threat than do experienced drivers (Scialfa et al., 2012). 


Experts Are Not Immune to Distraction 


Studies have shown that although expertise influences drivers’ ability to 
manage interference caused by secondary tasks, cellphone use worsens both 
expert and novice performance (Cooper & Strayer, 2008). For example, 
overall, expert drivers committed fewer errors than novice drivers, but all 
drivers committed more errors while using cellphones, regardless of expertise 
(Nabatilan, Aghazadeh, Nimbarte, Harvey, & Chowdhury, 2012). Another 
study also found that expert drivers (6 to 34 years driving) were more 
situationally aware and committed fewer driving infractions that novice 
drivers (unlicensed), but both groups suffered similar performance 
decrements while using a cellphone (Kass, Cole, & Stanny, 2007). 
Interestingly, the same study found that when using a cellphone, expert driver 
performance was equivalent to novice performance when the latter were not 
using cellphones (Kass et al., 2007). 

Experts and novices also differ in the frequency of engaging attention to 
secondary tasks, such as using a cellphone, over time. For example, in 
driving, novices (<3 weeks) engaged in secondary tasks more frequently as 
the length of the drive increased (Klauer et al., 2014). Expert drivers (average 
20 years), on the other hand, retained the same relative level of secondary 
task engagement across time. Klauer et al. suggest the possible explanation 


that as time driving increases, novices feel more confident in their driving. 


Expert Performance Differs Based on Type of 
Technology 


With the advent of more and more sophisticated technology, vehicle operator 
roles are looking much different than they did even just a few years ago. Self- 
driving cars are reducing the driver to a passenger merely monitoring the 
system. The Next Generation Air Transportation System (NextGen) is a 
multi-billion dollar initiative that is automating the air traffic controllers’ jobs 
as well as the flight crews’ jobs. Decision aids that generate multiple 
recommendations and solutions are changing the pilot to a manager of 
systems rather than an active agent in the system. These dramatic changes in 
the transportation domain may challenge an experienced vehicular operator 
who has not only general domain knowledge, but years of developing 
system-specific knowledge. 

For example, a recently tested aircraft attitude indicator (AI), the Arc- 
Segmented Attitude Reference (ASAR), was used by experienced pilots (a 
minimum of 100 flight hours) and they had a difficult time with the new 
technology (Wickens, Self, Andre, Reynolds, & Small, 2007). Although 


several display instruments in the cockpit are interpreted by the pilot to 


understand the attitude of an aircraft (climbing, descending, turning), the AI 
produces the greatest information about the attitude of an aircraft. Wickens et 
al. found that experienced pilots took twice as long as novices to recover 
from unusual attitudes when using the new ASAR technology. It is assumed 
that slower response time was due to negative transfer in experts because 


they had more experience with the traditional AI. Therefore, greater training 


for experienced vehicle operators may be necessary for new technology, 
especially if the functionality as well as the structure features do not match up 
with their traditional equipment. 

On the other hand, if technology is used to enhance information without 
major changes to the user’s mental model, experienced vehicle operators 
seem to improve to a greater extent than without the modification. 
Dijksterhuis and colleagues (Dijksterhuis, Stuiver, Mulder, Brookhuis, & de 


Waard, 2012) found that experienced drivers (average of about 7.5 years of 


driving experience) maintained better lane control in a driving simulator 
when an adaptive support system flashed a Head-up display that represented 
the status of the automobile than when the adaptive support system was not 
activated. In addition, participants stated that they felt the adaptive support 
systems were beneficial compared to not having a safety support system. 
Thus, modifications to systems that can be integrated into a legacy system 
with minimal perceptual and functional system changes would be more 


advantageous to the experienced vehicular operator 


Future Travel 


The world of transportation is likely to change considerably in the next ten 
years, both in the technology and in the age of the operators who use it. 
Although it is recognized and anticipated that technology is continuously 
becoming more sophisticated (e.g. computerized, digital displays, decision 
support tools), the technology on the horizon (e.g. self-driving cars), in the 
truest sense, is changing the vehicle operators’ role to a supervisory one. Will 
these technological changes (in some cases drastic changes) in vehicular 
operation interact with the experience of the operator? 

It is likely that younger, less experienced operators may have greater 
exposure to those types of technology changes than their older, more 
experienced counterparts may. Thus, young drivers might greatly benefit 
from driver support systems (Lee, 2007). 

The limited exposure to the rapidly changing technology may make it 
extremely difficult, at least initially, for more experienced and older operators 
to adapt to the new age of technology as seen in Wickens et al. (2007). New 
integrated, graphical and ecological interface design displays, such as the Oz 
display (see Eskridge, Still, & Hoffman, 2014) can be beneficial for 
improving a pilot’s understanding of a variety of aircraft situations and 
conditions (e.g. position, fuel management, performance). Once understood, 
these new technology displays can improve experienced pilots’ situation 
awareness, as well as improve student pilots’ understanding of the aircraft 
attitude (configuration) when compared to the traditional displays. However, 


the expediency of younger and less experienced operators adapting to new 


technology in comparison to older, more experienced operators does not 
compensate for the all-important mastery of knowledge and skills that the 
expert has developed. Thus, the challenge for our travel into the future is 
twofold. First, training for new technology needs to be sensitive to the age 
and experience of the operator. That is, agile schematic training protocols 
might need to be developed based on the characteristics, demographics, 
experiences, and proficiency levels of the training groups. Second, we need 
to integrate the functional assets that experienced operators possess into any 
training design. This integration and maintenance of the subtle and not so 
subtle assets that experts possess as vehicular operators to meet the 
challenges of the changing technological environment is the most critical 
aspect for moving forward in transportation. 

Further, we continue to believe that the leverage point for training is a 
cognitive one. Training should lead the novice to develop adaptive strategies, 
including visual search strategies. Deliberate practice on identifying cues that 
presage hazards seems like an especially promising direction for training. In 
addition to training to improve abilities, some awareness-raising about risk 


and distraction can certainly help both apprentice and expert drivers. 
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Introduction 


Studies of the nature of expert performance in professional design originated 
in the late 1960s with protocol studies of architects and other design 
professionals. Primarily, they have been conducted by researchers who are 
themselves situated within the design professions, and these studies have 
been an important element in the more general growth of design research. 
More recently, researchers from fields such as psychology and cognitive 
science have also begun to make significant contributions to the study of 
design expertise. 

I will review and give examples of the range of research methods that 
have been applied in developing the understanding of expertise in design. I 
will then discuss the key aspects of design expertise, and some of its apparent 
weaknesses, that have been established from these studies. I will briefly 
comment on some of the observed development of competence within 
students of design, and conclude with a summary of what we know of the 
nature of design expertise. 

My starting point is that the ability to design is widespread amongst all 


people, but that some people appear to be better designers than others. 


Design Ability 


Everyone can — and does — design. We all design when we plan for 
something new to happen, whether that might be a new version of a recipe, a 
new arrangement of the living room furniture, or a new layout of a personal 
web page. The evidence from different cultures around the world, and from 
designs created by children as well as by adults, suggests that everyone is 
capable of designing. So design capability is something inherent within 
human cognition; it is a key part of what makes us human. 

Human beings have a long history of design capability, as evidenced in 
the artifacts of previous civilizations and in the continuing traditions of 
vernacular design and traditional craftwork. Everything that we have around 
us has been designed. Anything that isn’t a simple, untouched piece of nature 
has been designed by someone. The quality of that design effort therefore 
profoundly affects our quality of life. The ability of designers to produce 
effective, efficient, imaginative, and stimulating designs is therefore 
important to all of us. 

To design and make things are normal activities for humans, and 
“design” has not always been regarded as something needing special abilities. 
Designing used to be regarded as a collective or shared capability, and it is 
only in fairly recent times that the ability to design has come to be regarded 
as a kind of exceptional individual talent. In traditional, craft-based 
production, the conception, or “designing,” of artifacts is not really separate 
from making them; that is to say, there is usually no prior activity of drawing 


or modeling before the activity of making the artifact. For example, a potter 


will make a pot by working directly with the clay, and without first making 
any sketches or drawings of the pot. In modern, industrial production, 
however, the activities of designing and of making artifacts are usually quite 
separate: the process of making something does not normally start before the 
process of designing it is complete. 

Ever since the emergence of design as a profession, it has appeared that 
some people have a level of design ability that is more highly developed than 
in other people. Design ability has been regarded as something that many 
people possess to some degree, but only a few people have a particularly 
strong design talent or “gift.” Of course, we know that some people are better 
designers than others. In fact, some people are very good at designing, and 
expertise is acknowledged through peer recognition, established reputations, 
and awards for innovative, high quality, or high performance designs. 
Although there is so much design activity going on in the world, the ways in 
which people design were rather poorly understood for rather a long time. 
However, there are now growing bodies of knowledge about the nature of 


designing, and about the core features or aspects of design expertise. 


Understanding Expertise in Design 


The study of expertise in design originated in the late 1960s with studies in 
architecture, and developed across a variety of other domains such as 
engineering, product, and software design (Cross, 2001). In these studies, 
participants have been categorized as novice designers (students at various 
stages of development), experienced designers (professional designers with 
several years of experience), expert designers (those with well-established 
reputations for excellence within their profession), or outstanding designers 
(those with international reputations and records of innovative and 
exceptional performance, including awards for high quality, or consistently 
high objective achievements in their field). As well as studies of individuals, 
there have also been studies of teams. Research methods have included 
interviewing expert and outstanding designers, ethnographic observation and 
case studies of professional design activity, experimental studies especially 
protocol studies of novice and experienced designers, neurological studies, 
and computational attempts to model or simulate expert performance. 
Interview studies have focused on designers who are acknowledged as 
having well-developed or outstanding design expertise, and have been based 
on conversations or interviews that sought to obtain these designers’ 
reflections on the processes and procedures they use, either in general, or 
with reference to particular works of design. Observations and case studies 
have tended to focus on one particular design project at a time, with 
observers recording the progress and development of the project either 


contemporaneously or post hoc. Both participant and non-participant 


observation methods have been included, and varieties of real, artificially 
constructed, and even reconstructed design projects have been studied. 
Experimental methods have usually been applied to artificial projects, related 
to the stringent requirements of recording the data. They include asking the 
participants to conduct a short design task and to “think aloud” as they 
respond to the task, and using these statements and the associated actions of 
the participants as the basis of protocol analysis studies. 

An interview study by Davies and Talbot (1987) was conducted with 
members of the UK-based Faculty of Royal Designers for Industry. This is an 
elite body of designers, across all domains of design practice, affiliated to the 
Royal Society for the Encouragement of Arts, Manufactures and Commerce, 
or the Royal Society of Arts (RSA) as it is more commonly known. Royal 
Designers for Industry are selected for the honour of appointment to the 
Faculty on the basis of their records of sustained excellence in design. 

What many of these outstanding designers suggested is that they find 
some aspects of their work appear to them to be natural, perhaps almost 
unconscious, ways of thinking. They believe that this “intuitive” way of 
thinking may be something that they inherently possess, or it may be 
something that they developed through their education. Making decisions, or 
generating proposals, in the design process is something that they feel relaxed 
about, and for which they feel no need to seek rational explanations or 
justifications. But it may be that they are overlooking the experience that they 
have gathered, and in fact their “intuitive” responses may be derived from 
these large pools of experience, and from prior learning gained from making 
appropriate, and inappropriate, responses in certain situations. 

What designers say about what they do can of course be rather biased, or 


based on partial recall, or limited by their willingness or ability to articulate 


what are complex cognitive activities. Sometimes, some designers can even 
seem to be willfully obscure about how they work, and where their ideas 
come from. The renowned designer Philippe Starck is known to have 
suggested that design ideas seem to come to him quite magically, as if from 
nowhere (Carmel-Arthur, 1999). For example, of the design process of his 
iconic lemon squeezer for the Italian kitchenware manufacturer Alessi, Starck 
said that, shortly after being set the task, whilst eating squid in a restaurant, 
“this vision of a squid-like lemon squeezer came upon me” (Carmel-Arthur, 
1999, p. 13). And so, Juicy Salif, the lemon squeezer, was conceived, went 
into production, and became a phenomenally successful product in terms of 
sales (if not necessarily in terms of its apparent function). 

But it is possible to construct a less mystical account of the conception 
of Starck’s lemon squeezer. In deconstructing this particular design act, 
Lloyd and Snelders (2003) utilized what Starck has said about himself in 
various interviews, what (little) he said about the conception of the lemon 
squeezer, and in particular the evidence of his very first design sketches for it, 
which were made on the restaurant placemat. Lloyd and Snelders implied that 
the “squid-like” concept was not an inexplicable flash of inspiration, but that 
it arose rather more prosaically by applying an analogy (which happened to 
be at hand, in the form of the squid) to the problem that was in the designer’s 
mind (to create a novel form for a lemon squeezer). The utility of this kind of 
analogy-making is often encountered in accounts of creative thinking (e.g. 
Boden, 1990). What was particularly striking in this case was Starck’s ability 
to make the mental leap of transposing “squid” to “lemon squeezer.” 
Thereafter, Lloyd and Snelders suggested, Starck, in developing the concept, 
was doing what many designers do, which is to draw upon a repertoire of 


precedents, of remembered images and recollections of other objects that 


helped him to give a more coherent, practicable, and attractive form to the 
concept. 

A more direct form of inquiry into understanding what designers do is 
actually to watch them at work, observing their activities. Bucciarelli (1994) 
conducted a series of participant-observer studies of engineering design 
projects in three different companies. Large projects demand an important 
aspect of design ability, that of reconciling the variety of interests — technical, 
financial, social, aesthetic, etc. — that inevitably have to coalesce around a 
major project. In these cases, designing becomes not just a_ personal, 
cognitive process, but a shared, social process. The main conclusion that 
Bucciarelli stressed is how even engineering design, traditionally seen as a 
strictly technical process, is in reality a social process of interaction and 
negotiation between the different participants who each bring to bear their 
own “object world” — their own specific knowledge and awareness of aspects 
of the object being designed. The social nature of designing, Bucciarelli 
suggested, requires acknowledging the inevitability of uncertainty and 
ambiguity, even within the process of engineering design. 

Lawson (1994) suggested that successful designers are good at coping 
with this uncertainty and ambiguity. From interviews with several 
outstanding architects, he identified their ability to maintain parallel 
processes of cognition relevant to the same design job at the same time, for 
example, working on detail junctions of materials at the same time as on 
general spatial concepts of a design. Lawson described how the architect 
Robert Venturi, working on a design for a major extension to the National 
Gallery in London, maintained one particular line of thought concerned with 
ideas for relating the circulation system in the new building to that in the 


older part (issues of the plan, and of floor levels), whilst another was for 


relating together the external appearances of the new and old parts (issues of 
the elevation, and of architectural styles). Lawson suggested that Venturi kept 
these two sets of ideas in progress, both equally important to his design 
thinking, before resolving them into a single solution. 

One way to cope with uncertainty is to try to impose order. Darke 
(1979) interviewed a number of expert architects, and noticed how they 
sought to impose order on the rather nebulous problems they faced. Some 
brought to the problem a personal set of guiding principles that offered 
starting points, some sought to find starting points in the particularities of the 
site on which they were to build. In each case, Darke observed how these 
Starting points enabled the designers to limit the problem to something 
manageable, to provide a narrower focus within which they could work. The 
designers imposed a limited set of objectives, or an idea about the building 
form, as a “primary generator,” as Darke called it, a means of instantiating a 
solution concept. This seems to be a necessary part of the design process, 
because a solution concept cannot be derived directly from the problem 
statement; the designer has to add or bring something to it. 

Early attempts at analyzing expert design behavior tended to borrow 
language and concepts from cognitive science studies of problem-solving 
behavior. However, gradually it became clear that designing is not “normal” 
problem-solving. Goel and Pirolli (1992), even from within a conventional 
cognitive science paradigm of problem-solving, successfully established 
designing as distinct from non-design problem-solving. 

However, design ability seems to be a natural part of human cognition, 
and there is evidence that such deep-seated strategies and competencies can 
be impacted and even lost through neurological damage in the brain. One of 


these cases was reported by Goel and Grafman (2000), who studied an 


architect who had had a seizure, associated with a meningioma tumor in his 
right prefrontal cortex. Before his attack, this person had _ practiced 
successfully as an architect. Through protocol analysis studies, Goel and 
Grafman compared this architect’s post-attack design ability with that of a 
healthy, matched control person (another architect with similar education and 
design experience), on being given a relatively simple task of redesigning a 
laboratory space. Both participants began by making a survey drawing of the 
existing laboratory and its furniture. The healthy control architect then 
produced a coherent series of sketches, beginning with abstracted 
considerations of circulation and organization, then developing proposals and 
refining the preferred one. The neurological patient produced three separate, 
basic, and incomplete proposals, finishing with a “final proposal” that was 
still inadequate and incomplete. 

The differences in the thinking processes of the two individuals became 
clear in graphs of the amount of time each devoted to different cognitive 
activities, as revealed by their think-aloud comments made during the 
experiments. The control architect focused initially on “problem structuring,” 
with periodical returns to this. He then moved to “preliminary design” and on 
to “refinement” and “detailing.” His graph clearly showed a controlled but 
complex pattern of activities, with overlap and quick transitions between 
activities. In contrast, the patient spent a huge amount of time on attempting 
“problem structuring,” and only small amounts of time on “preliminary 
design” and “refinement.” 

The experimenters reported that the patient understood the task, his 
architectural knowledge base was still intact, and he used it quite skillfully 
during the problem-structuring phase. However, he was unable to make the 


transition from problem-structuring to problem-solving, and simply could not 


perform the relatively simple design task. This unhappy case exposed some 
of the considerable complexity that there is in design thinking, and evidence 
that the normal brain has high-level cognitive functions that control or 
process activities that are essential aspects of design expertise. 

One neuroscience study (Alexiou, Zamenopoulos, Johnson, & Gilbert, 


2009; Gilbert, Zamenopoulos, Alexiou, & Johnson, 2009) investigated the 


neurological bases of design cognition using {MRI brain imaging techniques, 
with a mixed group of participants with varying experience in design. The 
researchers set the experiment participants a simple design task, involving the 
layout of furniture within a conference room, but two different versions of the 
problem task were given to different participant groups. One version was 
formulated as a well-defined problem-solving task, with specific constraints 
to be satisfied, such as “the two tables must face each other.” The other, less- 
defined and more design-like version, gave qualitative requirements, such as 
achieving a “spacious room” and a layout that “enables cooperation.” The 
findings suggested that the two conditions, problem-solving and designing, 
involve distinct cognitive functions associated with distinct brain networks, 
with the design condition associated with greater activity in right dorsolateral 
prefrontal cortex compared to the non-design condition. Such studies tend to 
confirm the view within the design studies community that there are 
particular, “designerly” ways of knowing, thinking, and acting (Cross, 2006), 


associated with identifiable, key aspects of design expertise. 


Key Aspects of Expertise in Design 


Conventional wisdom about the nature of expertise in problem-solving seems 
often to be contradicted by the behavior of expert designers. For example, 
expert designers challenge problem “rules” and often tackle the problem in a 
“difficult” way. Studies of design activity suggest that these unconventional 
features of design behavior actually may be the most effective and relevant to 
the intrinsic nature of designing. 

Unlike “normal” problem-solving, in a design project it is often not at all 
clear what “the problem” is; it may have been only loosely defined by the 
client, many constraints and criteria may be undefined, and everyone 
involved in the project may know that goals may be redefined during the 
project. So it is not unreasonable that in approaching a new project expert 
designers do not proceed by first attempting to define the problem rigorously. 
However, they do appear to have a number of characteristic key strategies or 


approaches for dealing with the given problems. 


Problem Framing 


Designers are not limited to “given” problems, but find and formulate 
problems within the broad context of the design brief. Processes of 
structuring and formulating the problem are frequently identified as key 
features of design expertise. The concept of “problem framing” seems to 
capture best the nature of this activity. Successful, experienced, and — 
especially — outstanding designers are found in various studies to be proactive 
in problem framing, actively imposing their view of the problem and 
directing the search for solution conjectures. 

This is also a more general characteristic of professional reflective 
practice identified by Schén (1983, p. 40) as problem setting: “Problem 
setting is the process in which, interactively, we name the things to which we 
will attend and frame the context in which we will attend to them.” This 
seems to characterize well what has been observed of the problem 
formulation aspects of design behavior. Designers select features of the 
problem space to which they choose to attend (naming) and identify areas of 
the solution space in which they choose to explore (framing). Schén (1988, 
p. 182) suggested that, “In order to formulate a design problem to be solved, 
the designer must frame a problematic design situation: set its boundaries, 
select particular things and relations for attention, and impose on the situation 
a coherence that guides subsequent moves.” 

This kind of problem framing has been noted often in studies of 


architects. Lloyd and Scott (1995), from their studies of architects, reported 


that “In each protocol there comes a time when the designer makes a 


statement that summarises how he or she sees the problem or, to be more 


specific, the structure of the situation that the problem presents” (p. 397). 
They referred to this “way of seeing the design situation” as the designer’s 
“problem paradigm.” In common also with their studies of engineers (Lloyd 


& Scott, 1994), they found that the architects who had specific prior 


experience of the problem type had different approaches from their less 
experienced colleagues: the experienced architects’ approaches were 
characterized by strong problem paradigms, or “guiding themes.” Cross and 
Clayburn Cross (1998) also identified, from interviews and protocol studies, 
the importance of problem framing, or the use of a strong guiding theme or 
principle, in the design behavior of expert and outstanding engineering 
designers. 

Schén (1988, p. 182) pointed out that “the work of framing is seldom 
done in one burst at the beginning of a design process.” This was confirmed 
in Goel and Pirolli’s (1992) protocol studies of several types of designers 
(architects, engineers, and instructional designers). They found that “problem 
structuring” activities not only dominated at the beginning of the design task, 


but also reoccurred periodically throughout the task. 


Solution Conjecturing 


Experience within a problem domain enables designers to move quickly to 
identifying a problem frame and proposing a solution conjecture. This 
appears to be a feature of design cognition that comes with experience in 
designing. Generating a wide range of alternative solution concepts is 
frequently recommended by theorists and educationists but appears not to be 
normal in expert design practice. Generating a very wide range of alternatives 
may not be a good thing: some studies have suggested that a relatively 
limited amount of generation of alternatives may be the most appropriate 
strategy. 

The solution-focused nature of designing was first noted by Lawson 
(1979) in comparisons of problem-solving behavior by scientists and 
architects. Subsequently, many studies of expert design behavior suggest that 
designers move rapidly to early solution conjectures, and use these 
conjectures as a way of exploring and defining problem-and-solution 


together. Lloyd and Scott (1994), from their protocol analysis studies of 


experienced engineering designers, found that this solution-focused approach 
appeared to be related to the degree and type of previous experience of the 
designers. They found that more experienced designers used more 
“generative” reasoning, in contrast to the deductive reasoning employed more 
by less experienced designers. In particular, designers with specific 
experience of the problem type tended to approach the design task through 
solution conjectures, rather than through problem analysis. They concluded 


that “It is the variable of specific experience of the problem type that enables 


designers to adopt a conjectural approach to designing, that of framing or 
perceiving design problems in terms of relevant solutions” (p. 140). 

This aspect of design expertise has been noted even from the very 
earliest formal studies. Eastman (1970), in the earliest recorded design 
protocol study (of experienced architects), found that: “One approach to the 
problem was consistently expressed in all protocols. Instead of generating 
abstract relationships and attributes, then deriving the appropriate object to be 
considered, the subjects always generated a design element and then 
determined its qualities” (p. 27). That is to say, the designers jumped to ideas 
for solutions (or partial solutions) rather than attempting first to fully 
formulate the problem. This is a reflection of the fact that designers are 
solution-led, not problem-led; for designers, it is the evaluation of the 
solution that is important, not the analysis of the problem. 

It is not just that problem analysis is weak in design; even when problem 
goals and constraints are known or defined, they are not sacrosanct, and 
designers exercise the freedom to change goals and constraints, as 
understanding of the problem develops and definition of the solution 
proceeds. As Ullman, Dietterich, and Stauffer (1988) pointed out, from their 
protocol studies of experienced mechanical engineering designers, only some 
constraints are “given” in a design problem; other constraints are 
“introduced” by the designer from domain knowledge, and others are 
“derived” by the designer during the exploration of particular solution 


concepts. 


Co-evolving Problem-Solution 


Designers tend to use solution conjectures as the means of developing their 
understanding of the problem. Since “the problem” cannot be fully 
understood in isolation from consideration of “the solution,” it is natural that 
solution conjectures should be used as a means of helping to explore and 
understand the problem formulation. 

Creative design is not a matter of first fixing the problem and then 
searching for a satisfactory solution concept; instead, it seems more to be a 
matter of developing and refining together both the formulation of the 
problem and ideas for its solution, with constant iteration of analysis, 
synthesis, and evaluation processes between the two “spaces” of problem and 
solution. As Kolodner and Wills (1996, p. 390) observed, from a study of 


engineering designers: “Proposed solutions often directly remind designers of 


issues to consider. The problem and solution co-evolve.” The concept of “co- 
evolution” was introduced in computer modeling of design processes by 
Maher (1994), and has been developed to describe how designers develop 
aspects of both the problem and the solution together in conceptual stages of 
the design process. 

In this interpretation of design as a co-evolution of both solution and 
problem, the designer’s attention oscillates between the two, forming partial, 
related structurings within the problem and solution spaces. Dorst and Cross 
(2001) observed this behavior in protocol studies of experienced industrial 
designers. They reported that: “The designers start by exploring the [problem 
space], and find, discover, or recognise a partial structure. That partial 


structure is then used to provide them also with a partial structuring of the 


[solution space]. They consider the implications of the partial structure within 
the [solution space], use it to generate some initial ideas for the form of a 
design concept, and so extend and develop the partial structuring ... They 
transfer the developed partial structure back into the [problem space], and 
again consider implications and extend the structuring of the [problem space]. 
Their goal ... is to create a matching problem-solution pair” (pp. 434-435). 

Wiltschnig, Christensen, and Ball (2013) extended the study of co- 
evolution from laboratory studies of individual designers into team design 
processes in professional engineering design practice. They found that co- 
evolution was significantly present in such practice, identifying episodes of 
co-evolution that demonstrated elements from within the problem space 
being intimately linked with solution generation activity. These episodes 
revealed close links between co-evolution and creative processes, and they 
concluded that “the evidence points to co-evolution episodes as being the 
creative engine of everyday design practice” (p. 539). 

During a co-evolutionary design process, partial models of the problem 
and solution are constructed side-by-side, as it were. But the crucial factor, 
often regarded as a “creative leap,” is the bridging of these two partial models 
by the articulation of a concept that enables the models to be mapped onto 
each other. Cross (1997) argued that, in design, the creative event is not so 
much a “creative leap” from problem to solution as the building of a “bridge” 
between the problem space and the solution space by the identification of a 
key concept. The development of a satisfactory bridging concept or 
conjecture embodies constructive relationships between problem and 
solution. It is the recognition of such a satisfactory concept that provides the 
illumination of the creative “flash of insight.” The interdependent 


development of problem and solution spaces, and the recognition of a 


satisfactory bridging concept for a problem-—solution pair, are considered key 
features that characterize creative design as a process of exploration rather 
than search. Creative design involves a period of exploration in which 
problem and solution spaces are evolving and are unstable until (temporarily) 
fixed by an emergent problem-—solution pair. 

A related aspect of cognitive strategy that emerges from some studies is 
that, especially during creative periods of conceptual design, expert designers 
alternate rapidly in shifts of attention between different aspects of their task, 
or between different modes of activity. Akin and Lin (1996), in a protocol 
study of an expert engineering designer, first identified the occurrence of a 
sequence of novel design decisions. These, in contrast to routine design 
decisions, are decisions that are critical to the development of the design 
concept. Akin and Lin also segmented the designer’s activities into three 
modes: drawing, examining, and thinking. Then, allowing for some implicit 
overlap or carry-over of the designer’s attention from one segment to another, 
they represented the designer’s activities in terms of single-, dual-, or triple- 
mode periods. They found a significant correlation between the triple-mode 
periods and the occurrence of the novel design decisions, where the designer 
was alternating between these three activity modes 
(examining—drawing-—thinking) in rapid succession. Akin and Lin were 
cautious about drawing any inference of causality, concluding only that “Our 
data suggest that designers explore their domain of ideas in a variety of 
activity modes ... when they go beyond routine decisions and achieve design 
breakthroughs” (p. 59). 

Several of these features of expert designer behavior were confirmed 
and clarified by Suwa, Gero, and Purcell (2000) from a protocol study of an 


experienced architect. They concentrated on the occurrence of “unexpected 


discoveries” during the design process — that is, those instances when a 
designer perceives something “new” in a previously drawn element of a 
solution concept — and related these to the “invention” of further issues or 
requirements within the design problem. They found a strong bidirectional 
correlation between unexpected discoveries and the invention of issues and 
requirements. Suwa, Gero, and Purcell suggested that their findings provided 
empirical evidence both for the co-evolution of problem space and solution 
space and for designing as a “situated” act — that is, designers invent design 
issues or requirements in a way situated in the environment in which they 
design. Their analysis also confirmed the importance of rapid alternation 
between different modes of activity, facilitated by external representations: 
“drawing sketches, representing the visual field in the sketches, perceiving 
visuo-spatial features in sketches, and conceiving of design issues or 
requirements are all dynamically coupled with each other” (Suwa et al., 2000, 
p. 564). 


Representations 


A key tool to assist design cognition is the use of representations, including 
the traditional form of the sketch. It seems to support and facilitate the 
uncertain, ambiguous, and exploratory nature of conceptual design activity. 
Sketching is tied in very closely with features of design cognition such as the 
generation and exploration of tentative solution concepts, the identification of 
what needs to be known about the developing concept, and especially the 
recognition of emergent features and properties. 

One obvious purpose of sketching and drawing is that the end point of 
the design process usually requires a drawing, or a set of drawings, that 
provides a model of the object that is to be made by the builder or 
manufacturer. That is the designer’s goal — to provide that model. If, given 
the brief for a new product, the designer could immediately make that final 
model, then there would really be no need for a design process at all — the 
designer would simply read the brief and then prepare the final drawings. 

It seems that designing is difficult to conduct by purely internal mental 
processes; the designer needs to interact with an external representation. The 
activity of sketching, drawing, or modeling provides some of the 
circumstances by which designers put themselves into the design situation 
and engage with the exploration of both the problem and its solution. There is 
a cognitive limit to the amount of complexity that can be handled internally; 
sketching provides a temporary, external store for tentative ideas, and 
supports the ongoing “dialogue” that the designer conducts between problem 


and solution. 


Several researchers have referred to the ways in which sketching helps 
to assist cognition in design thinking. Sketching helps the designer to find 
unintended consequences, the surprises that keep the process of design 
exploration going in what Sch6én and Wiggins (1992) called the “reflective 
conversation with the situation” that is characteristic of design thinking. 
Goldschmidt (1991) called it the “dialectics of sketching”: a dialogue 
between “seeing that” and “seeing as,” where “seeing that” is reflective 
criticism and “seeing as” is the analogical reasoning and reinterpretation of 
the sketch that provokes creativity. Goel (1995) suggested that sketches help 
the designer to make not only “vertical transformations” in the sequential 
development of a design concept, but also “lateral transformations” within 
the solution space: the creative shift to new alternatives. Goel referred 
especially to the ambiguity inherent in sketches, and identified this as a 
positive feature of the sketch as a design tool. 

It is not just formal or shape aspects of the design concept that are 
compiled by sketching; they also help the designer to identify and consider 
functional and other aspects of the design. Suwa, Purcell, and Gero (1998) 
suggested that sketching serves at least three purposes: as an external 
memory device in which to leave ideas as visual tokens, as a source of 
visuospatial cues for the association of functional issues, and as a physical 
setting in which design thoughts are constructed in a type of situated action. 
Although the above studies refer mostly to sketching in architectural design, 
Ullman, Wood, and Craig (1990) also studied and emphasized the importance 
of sketching in mechanical engineering design, as have others in respect of 
various domains of design practice. The critical, reflective dialogue through 
sketching seems to be relevant in all forms of design. 


Clearly the use of external representations is something important in the 


design process. One reason is that sketches enable designers to handle 
different levels of abstraction simultaneously. As various studies have shown, 
during design activity expert designers are thinking about the overall concept 
and at the same time thinking about detailed aspects of the implementation of 
that concept. Obviously not all of the detailed aspects are considered early 
on, because if the designers could do that, they could go straight to the final 
set of detailed drawings. So they use sketches to identify and then to reflect 
upon critical details — details that they realize will hinder, or somehow 
significantly influence the final implementation of the detailed design. This 
implies that, although there may be a hierarchical structure of decisions, from 
overall concept to details, designing is not a strictly hierarchical process; in 
the early stages of design, the designer moves freely between different levels 
of detail. 

The identification of critical details is part of a more general facility that 
sketches provide, which is that they enable identification and recall of 
relevant knowledge. There is a massive amount of information that may be 
relevant, not only to all the possible solutions, but simply to any possible 
solution. And any possible solution in itself creates the unique circumstances 
in which these large bodies of information interact, probably in unique ways 
for any one possible solution. So these large amounts of information and 
knowledge need to be brought into play in a selective way, being selected 
only when they become relevant, as the designer considers the implications 
of the solution concept as it develops. 

Another key benefit of design sketches is that they assist problem 
structuring through solution attempts. Designers’ sketches incorporate not 
only drawings of tentative solution concepts but also numbers, symbols, and 


texts, as designers relate what they know of the design problem to what is 


emerging as a solution. Sketching enables exploration of the problem space 
and the solution space to proceed together, assisting the designer to converge 
on a matching problem-solution pair. It enables exploration of constraints 
and requirements, in terms of both the limits and the possibilities of the 
problem and solution spaces. Perhaps crucially, sketches in design promote 


the recognition of emergent features and properties of the solution concept. 


Precedents 


In a co-evolutionary process of exploration, how do designers identify or 
construct partial structures within the problem and solution spaces? From 
protocol studies, Dorst and Cross (2001) suggested that expert designers seek 
to gather pieces of related information within the problem space. Expert 
designers appear to have a search strategy, pursuing a quasi-standard set of 
questions, and actively seeking or creating patterns within the data. They also 
have a store of knowledge of solution precedents, either from their own 
previous experience or from a broad knowledge of the domain in which they 
practice. Somewhat like chess players, expert designers seem to recognize 
patterns in problem situations and draw upon knowledge of precedents that 
enable them to build conceptual bridges between the problem and solution 
spaces. However, in designing, unlike chess playing, it is not possible simply 
to reproduce previous solutions; the new design in some way has to be 
original. 

In developing a knowledge-based computational model of design, 
Oxman (1990) considered the apparent paradox of how knowledge of 
precedents, which by definition are examples of the old, can be used to help 
generate proposals for the new. She suggested that a fundamental factor is the 
classification of prior solutions in terms of more abstract and generalized 
knowledge, the ability to progress from iconic sources to generic types. When 
designers study prior examples, therefore, it is not simply to copy them, but 
to abstract general principles of how to resolve similar situations. This is not 
the use of precedent in the form of a rule, or a normalized prior solution, but 


to adapt and extract from it in order to reach a creative reinterpretation. 


So a key competency of an expert designer, like other experts, is the 
ability mentally to stand back from the specifics of accumulated examples, 
and form more abstract conceptualizations pertinent to their domain of 
expertise. Experts are able to store and access information in larger cognitive 
chunks than non-experts, and to recognize underlying principles, rather than 
focusing on the surface features of problems. Lawson (2004) suggested that 
designers use knowledge of precedents that they have abstracted into solution 
chunks, or “schemata.” A typical design schema for an architect might be a 
way of organizing internal space, such as around an atrium, or for an 
industrial designer might be grouping together different functional parts of a 
product. Lawson also suggested that, like chess masters, expert designers 
have repertoires of “gambits,” or ways of proceeding, of entering into and 
opening up the problem situation. These can be guiding principles or the use 
of techniques such as geometrical formulations. Some designers use the same 
or similar “gambits” frequently, developing them into recognizable personal 


design styles. 


Apparent Weaknesses 


Several studies have shown that the normal practices of expert designers can 
exhibit certain features that might be regarded as weaknesses in their methods 
or approaches. One particular weakness appears to be their attachment to 
early solution ideas and concepts. Although designers change goals and 
constraints as they design, they appear to hang on to their principal solution 
concept for as long as possible, even when detailed development of the 
scheme throws up unexpected difficulties and shortcomings in the solution 
concept. Some of the changing of goals and constraints during designing is 
associated with resolving such difficulties without having to start again with a 
major new concept. For example, from case studies of professional 
architectural design, Rowe (1987, p. 36) observed that “A dominant influence 
is exerted by initial design ideas on subsequent problem-solving directions ... 
Even when severe problems are encountered, a considerable effort is made to 
make the initial idea work, rather than to stand back and adopt a fresh point 
of departure.” The same phenomenon was observed by Ullman et al. (1988) 
in protocol studies of experienced mechanical engineering designers. They 
found that “designers typically pursue only a single design proposal,” and 
that “there were many cases where major problems had been identified in a 
proposal and yet the designer preferred to apply patches rather than to reject 
the proposal outright and develop a better one” (p. 47). 

This apparent weakness may be associated with the “cognitive cost” of 
developing new design concepts, but might also be a positive aspect of expert 


behavior, related to the establishment of a “problem frame.” Cross and 


Clayburn Cross (1998) reported that expert designers can be tenacious in 
their pursuit of solution concepts that fit the frame, or guiding theme or 
principle that they adopted. Crilly (2015) suggested that experienced 
designers draw on their knowledge of prior projects but seek to maintain a 
balance between openness and a persistence in pursuing a solution concept. 

Another weakness has been suggested where details of known previous 
design solutions are carried over, perhaps inappropriately and even 
unconsciously, into new solutions. This “fixation” effect in design was 
suggested by Jansson and Smith (1991), who studied experienced mechanical 
engineers’ solution responses to design problems. They compared groups of 
participants who were given a simple, written design brief, with those that 
were given the same brief but with the addition of an illustration of an 
existing solution to the set problem. They found that the latter groups 
appeared to be “fixated” by the example design, producing solutions that 
contained many more features from the example design than did the solutions 
produced by the control groups. Jansson and Smith proposed that such 
fixation could hinder conceptual design if it prevents the designer from 
considering all of the relevant knowledge and experience that should be 
brought to bear on a problem. 

Purcell and Gero (1992, 1996) undertook a series of experiments to 
verify and extend Jansson and Smith’s findings on fixation. They studied and 
compared senior students in mechanical engineering and in industrial design. 
Their results suggested that the engineering students appeared to be much 
more susceptible to fixation than did the industrial design students; the 
engineers’ designs were substantially influenced by prior example designs, 
whereas the industrial designers appeared to be more fluent in producing a 


greater variety of designs, uninfluenced by examples. Purcell and Gero 


suggested that this might be a feature of the different educational programs of 
engineers and designers, with the latter being more encouraged to generate 
diverse design solutions. In a further development of the study, however, 
Purcell and Gero explored engineers’ and designers’ responses when the 
example design was either an innovative or a routine prior solution. Here they 
found that engineers became fixated in the traditional sense when shown a 
routine solution, i.e. incorporating features of the routine solution in their 
own solutions, but became fixated on the underlying principle of the 
innovative solution, i.e. producing new, innovative designs embodying the 
same principle. The industrial designers, however, responded in similar ways 
under both conditions, generating wide varieties of designs that were not 
substantially influenced by any of the prior designs. Purcell and Gero 
therefore concluded that the industrial designers seemed to be fixated on 
“being different.” 

Many further studies have been made of the use of examples to induce 
either fixation or inspiration. Sio, Kotovsky, and Cagan (2015) conducted a 
meta-analysis of such studies and concluded that being presented with 
examples can have a negative effect on the variety of ideas generated but a 
positive effect on the novelty and quality of ideas. They suggested that “the 
presence of examples can modify the search strategy from a broad one to a 
focused one. Although this narrows the scope of search, it allows a more in- 
depth exploration, and in turn, improves solution quality” (p. 91). These 
results are generally controlled experiments to investigate particular effects, 
whereas in practice, designers can choose and use examples, or precedents, 
more deliberately. 

An interview study of expert designers by Crilly (2015) focused on 


trying to understand their awareness of and attitudes toward fixation. He 


concluded that fixation is a real problem in a variety of ways in professional 
design practice, but that expert designers are aware of these problems and 
they take steps to address them, including discussion with colleagues and the 
building of models and prototypes. Cross and Clayburn Cross (1998) 
suggested that, in innovative design, expert designers are aware of the need to 
“keep experience at the back of [their] mind, not the front” (p. 148), and work 
from first principles, rather than copying a standard solution. 

Another apparent weakness in expert design behavior emerged from 
some studies that recorded the “opportunistic” behavior of designers. This 
emphasis has been on designers’ deviations from a structured plan or 
methodical process into the “opportunistic” pursuit of issues or partial 
solutions that catch the designer’s attention. Visser (1990) conducted a 
longitudinal study of an experienced mechanical engineer pursuing a design 
project. The engineer claimed to be following a structured approach, but 
Visser found frequent deviations from this plan, observing that “The engineer 
had a hierarchically structured plan for his activity, but he used it in an 
opportunistic way. He used it only as long as it was profitable from the point 
of view of cognitive cost. If more economical cognitive actions arose, he 
abandoned it” (p. 276). Thus Visser regarded reducing cognitive cost, i.e. the 
cognitive load of maintaining a principled, structured approach, as a major 
reason for abandoning planned actions and instead delving into, for example, 
confirming a partial solution at a relatively early stage of the process. 

From protocol analysis studies of three experienced software system 
designers, Guindon (1990) also emphasized the “opportunistic” nature of 
design activities. Guindon stressed that “designers frequently deviate from a 
top-down approach. These results cannot be accounted for by a model of the 


design process where problem specification and understanding precedes 


solution development and where the design solution is elaborated at 
successively greater levels of detail in a top-down manner” (p. 326). Guindon 
observed the co-evolution approach of interleaving problem specification 
with solution development, “drifting” through partial solution development, 
and jumping into exploring suddenly recognized partial solutions, which she 
categorized as major causes of “opportunistic solution development.” She 
also referred to cognitive cost as one possible explanation for such behavior. 

Ball and Ormerod (1995) criticized a too-eager willingness to emphasize 
“opportunism” in design activity. In their studies of expert electronics 
engineers they found very few deviations from a top-down, breadth-first 
design strategy. But they did find some significant deviations occurring, 
when designers made a rapid depth-first exploration of a solution concept in 
order to assess its viability. Ball and Ormerod did not regard such occasional 
depth-first explorations as implying the abandonment of a structured 
approach. Instead, they suggested that expert designers will normally use a 
mixture of breadth-first and depth-first approaches: “Much of what has been 
described as opportunistic behavior sits comfortably within a structured top- 
down design framework in which designers alternate between breadth-first 
and depth-first modes” (p. 148). Ball and Ormerod were concerned that 
“opportunism” seemed to imply unprincipled design behavior, a non- 
systematic and heterarchical process in contrast to the assumed ideal of a 
systematic and hierarchical process. 

However, rather than regarding opportunism as unprincipled design 
behavior, Guindon (1990) suggested it might be inevitable in design: “These 
deviations are not special cases due to bad design habits or performance 
breakdowns but are, rather, a natural consequence of the ill-structuredness of 


problems in the early stages of design” (p. 307). So it may be that we should 


not equate “opportunistic” with “unprincipled” behavior in design, but rather 
that we should regard “opportunism” as a characteristic of expert design 
behavior. 

These examples do seem to suggest some weak features in expert design 
behavior. However, trying to change the “unprincipled” and “ill-behaved” 
nature of conventional design activity may be working against aspects that 


are actually effective and productive features of design expertise. 


Developing Expertise in Design 


Education in design has well-established practices, predominantly project- 
and studio-based learning, that are assumed to help students develop in a 
progression from novice toward expert; but these are not very well 
understood, and certainly not well researched, documented, and explained. 
There is still rather limited understanding of the differences between novice 
and expert performance in design, and in particular how to help students 
move from one to the other. 

In studies of problem-solving, novice behavior is usually associated with 
a depth-first approach, whilst the strategies of experts are usually regarded as 
exhibiting predominantly breadth-first approaches. Differences of this nature 
were found between the behavior of novice and experienced designers by 
Ahmed, Wallace, and Blessing (2003). They found clear differences between 
the behaviors of new (graduate) entrants to the engineering profession and 
much more experienced engineers. The new entrants used trial-and-error 
techniques of generating and implementing a design modification, evaluating 
it, then generating another, and so on through many iterations. Experienced 
engineers were observed to make a preliminary evaluation of their tentative 
decisions before implementing them and making a final evaluation. They 
used the foresight they had gained from experience to consider whether it 
seemed worthwhile to move further into the implementation stage of a design 
decision. 

Developing greater expertise generally means developing a broader and 


more complex understanding of what has to be achieved, and this is also 


common in developing design expertise. For example, studies of novice 
(graduate student) and expert designers in the field of woven textiles found 
that the novices concentrated on the visual composition task and only 
occasionally moved to construction issues to explore how the visual ideas 
could be realized in the weaving. In contrast, the experts integrated both the 
visual and the technical elements of weaving, and generally considered them 
in a parallel way during the design process (Seitamaa-Hakkarainen & 
Hakkarainen, 2001). 

Christiaans and Dorst (1992) found, from protocol analysis studies of 
junior and senior industrial design students, that some students became stuck 
on information gathering, rather than progressing to solution generation. 
They found that this was not such a significant difficulty for junior students, 
who did not gather a lot of information, and tended to “solve a simple 
problem,” being unaware of a lot of potential criteria and difficulties. But 
they found that senior students could be divided into two types. The more 
successful group, in terms of the quality of their solutions, “asks less 
information, processes it instantly, and gives the impression of consciously 
building up an image of the problem. They look for and make priorities early 
on in the process” (p. 135). This is the activity of problem framing. The other 
group gathered lots of information, but for them, the activity of naming, or 
simply gathering data, was sometimes just a substitute activity for actually 
doing any design work. 

In studies of junior versus senior student design behaviors, Adams, 
Tums, and Atman (2003) found that changes in individual students’ 
behaviors over the three or four years of their studies were quite complex and 
variable. Although there were identifiable changes in behavior for many of 


the students, some did not appear to change their behaviors at all and some 


seniors simply spent more time on the given design projects but without any 
qualitative behavioral changes. It also appeared that some students exhibited 
different behavioral changes for different types of design projects; they were 
perhaps on the cusp of development from beginners to more competent 
practitioners, showing more sensitivity to different problem situations. 

The development of expertise passes through different stages. 
Something happens in the development from novice to expert, involving a 
progression through different levels of ability. Thus a novice undergoes 
training and education in their chosen field, enters into practice, achieves 
some competence, and then at some later point becomes regarded as an 
expert. In all fields, the accumulation of experience is a vital part of the 
transformation to expert. 

Dreyfus and Dreyfus (2005) outlined a five-stage general model of skill 
acquisition and the development of expertise: Novice; Advanced Beginner; 
Competence; Proficiency; Expertise. Lawson and Dorst (2009) expanded this 
model to fit the development of expertise in design. They inserted a student 
“beginner” phase between novice and advanced beginner, acknowledging 
that most people can be “novice designers.” Also, the graduate student may 
have reached a level of competence, but probably is still an “advanced 
beginner” and needs much more practice and experience to become an expert. 
For most designers, the expert level of achievement is where they remain, 
perhaps with some continued moderate improvement. But some progress 
beyond the peak level of their peers, and enter into a further phase of 
development, reaching outstanding levels of achievement and eminence. 
They can even develop into “visionaries” who introduce completely new 


concepts and constructs to the profession. 


Conclusion 


Many of the classic studies of expertise have been based on examples of 
game-playing (such as chess), or on comparisons of experts versus novices in 
solving routine problems (e.g. in physics). These studies are generally 
examples of solving well-defined problems, whereas designers 
characteristically deal with ill-defined problems, where there is no definitive 
formulation of the problem, there are no rules of how to operate, and no 
single or verifiably correct solution. Some of the standard results from studies 
of expertise do not match with results from studies of expertise in creative 
domains such as design. For example, creative experts might reformulate the 
given task so that it is more problematic — i.e. deliberately treat it as ill- 
defined — which is contrary to the assumption that experts will generally 
solve a problem in the “easiest” way, or certainly with more ease than 
novices. In some ways, therefore, creative experts treat problems as “harder” 
problems than novices do. 

Expert designers appear to be “ill-behaved” problem-solvers, e.g. in that 
they do not devote extensive time and attention to defining the problem. 
However, it seems that this may well be appropriate behavior, since some 
studies have suggested that over-concentration on problem definition does 
not necessarily lead to successful design outcomes. It appears that successful 
design behavior is based not on extensive problem analysis, but on adequate 
“problem scoping” and on a focused or directed approach to gathering 


problem information, prioritizing criteria and generating solution concepts. 


Setting and changing goals, rather than sticking to the problem as given, are 
inherent elements of design activity. 
Expert designers perform in ways akin to other professionals operating 


in fields of naturalistic decision making (Klein, 1999), dealing with practical 


situations of uncertainty, inadequate information, and unclear goals. Klein 
suggested that experts in these situations make “recognition-primed” 
decisions: “They understand what types of goals make sense (so the priorities 
are set), which cues are important (so there is not an overload of 
information), what to expect next (so they can prepare themselves and notice 
surprises), and the typical ways of responding in a given situation. By 
recognizing a situation as typical, they also recognize a course of action 
likely to succeed” (p. 24). This description is very similar to how we have 
seen that designers operate. Like other professional decision-makers, expert 
designers do not work from “intuition” but have recognizable and appropriate 
strategies for dealing with their ill-defined problems, as research in 


understanding design expertise has shown. 
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Most people never pick up the phone and call. Most people never ask. 
And that’s what sometimes separates people who do things from the 
people who just dream about them. 


(Steve Jobs in an interview with Santa Clara Valley Historical 


Association, 1994) 


Introduction 


This chapter reviews research on expertise in entrepreneurship. Over the past 
two decades researchers have studied expert performance in numerous 
professional and organizational domains (e.g. teaching, software, medicine, 
taxi driving), extending expertise investigations beyond traditional studies in 
games, sports, and the arts. These streams of literature support the hypothesis 
that expertise develops and is sustained through both purposeful and 
deliberate practice in a domain. As Ericsson and Pool (2016) define the 
terms, whereas naive practice may consist in nothing more than doing 
something repeatedly, purposeful practice is more focused on continual 
improvement by repeatedly engaging in practice tasks with immediate 
feedback. Purposeful practice is also more effortful in pushing one out of 
one’s comfort zone. Deliberate practice goes one step further than purposeful 
practice in that it requires supervision from a trained teacher. In 
entrepreneurship research there is a growing body of work demonstrating the 
existence of expertise. However, only recently have explicit mechanisms of 
purposeful practice been proposed and been subject to study. In this chapter 
we first review over two decades of scholarship on entrepreneurial expertise 
and then outline our own work that has posited “the Ask” as an important 
mechanism for purposeful practice in entrepreneurship that can and should be 
studied if we are to develop entrepreneurship education capable of fostering 
deliberate practice. The chapter closes with key implications from 
entrepreneurial expertise for expertise research more generally, and outlines 


an agenda for future research on expert entrepreneurial performance. 


A Brief History of Entrepreneurship Research 
Leading up to a Focus on Expertise 


Entrepreneurship research is relatively young, with empirical work 
commencing only in the 1980s. Although some theories of entrepreneurship 
can be traced back to economists in the first half of the twentieth century and 
even earlier (Cantillon, 1959 [c.1730]; Knight, 1921; Mises, 1949; 


Schumpeter, 1934), most empirical work before the 1980s consisted of 


informal case studies and surveys and interviews of dubious rigor. A few 
studies used formal economic and econometric models, but narrowly focused 
on occupational choice (Banerjee & Newman, 1993; Ginzberg, Ginsburg, 
Axelrad, & Herma, 1951; Parker, 2009). This research modeled the choice 


between employment and entrepreneurship as one driven by individual-level 


variables such as risk propensity (Kihlstrom & Laffont, 1979). 

Risk has been a preoccupation of economists all the way back to the 
very origin of the term “entrepreneur” in which Cantillon (1959 [c.1730]) 
pronounced the entrepreneur to be an individual who assumes the risk of 
running a firm. Another variable modeled in occupational choice research 
was liquidity constraints (i.e. the availability of funding) (Evans & Jovanovic, 


1989; Minniti & Lévesque, 2008). However, subsequent research showed 


that risk preferences have either limited or no impact on this career choice 
(Miner & Raju, 2004; Zhao, Seibert, & Lumpkin, 2010) and capital 
constraints have no impact at all (Cressy, 2002; Hurst & Lusardi, 2004; Kim, 


Aldrich, & Keister, 2006). Instead, studies show that whereas entrepreneurs 


self-report being risk tolerant, objective data strongly suggest they are risk 


avoidant (Sarasvathy, Simon, & Lave, 1998; Simons & Astebro, 2010). 


Additionally, because startups can be bootstrapped on budgets to suit every 
wallet (Bhide, 1991) and banks are increasingly forthcoming with credit (de 
Meza & Webb, 1987), access to capital proves not to be a major constraint. 

Given inconclusive results, recent research has acknowledged the need 
to examine occupational choice in conjunction with various other factors 
such as motivations and resources of the entrepreneur (Lévesque, Shepherd, 
& Douglas, 2002). Moreover, as Astebro, Chen, and Thompson (2011) have 
shown, at least a third of entrepreneurs do not make this choice at all. Instead 
these “hybrid” entrepreneurs start new ventures while continuing to work full 
time or part time in the labor market. Finally, research on entrepreneurial 
expertise offers specific insights into occupational choice, such as the 
affordable loss principle we elaborate upon later in this chapter (Dew, 
Sarasathy, Read, & Wiltbank, 2009). 

Pioneered by McClelland (1965), another important body of work that 
developed early in entrepreneurship research concerned individual 
psychological traits associated with entry into entrepreneurship and 
subsequent success. McClelland argued that one reason individuals pursue 
entrepreneurial careers has to do with their desire to achieve something, a 
construct he termed “nAch” (need for Achievement). Furthermore, he argued 
that entrepreneurship may better satisfy this need than other career choices. 
McClelland’s work marked the beginning of a research effort to decode the 
entrepreneurial personality profile. Dozens of personality variables were 
researched as part of the “traits” approach. However, by the late 1980s, a 
consensus emerged from reviews of this work that relationships between 
personality and entrepreneurship are inconsistent and weak, leading to calls 


to abandon the traits approach (Brockhaus & Horwitz, 1986; Gartner, 1988). 


Yet this conclusion proved premature, even though it hung over the field 
for two decades until a new generation of researchers began to revisit prior 
studies of personality using meta-analytic methodology (Zhao & Seibert, 
2006). Reanalyzing prior results led to findings in favor of modest 
relationships between certain personality variables and new business creation 
and success. Among those variables are nAch (Stewart & Roth, 2007) and 
openness to experience (Zhao et al., 2010). The latter, in particular, highlights 
the notion that entrepreneurial success may be enabled by dispositions 
favoring new ideas, creativity, unconventional values, and learning. 

In addition to the economics of occupational choice and the psychology 
of entrepreneurs, a third stream of work in entrepreneurship examined the 
role of knowledge and skills. Early papers in this stream drew on Gary 
Becker’s (1964) concept of “human capital.” Initial studies suggested that 
since many entrepreneurs had little formal education, the skills and 
knowledge needed to start up and manage new ventures must be developed 
from on-the-job experience (Cooper, Gimeno-Gascon, & Woo, 1994; Reuber 
& Fischer, 1994). Through experience, entrepreneurs accumulate knowledge 
about products, technologies, markets, customers, investors and suppliers, 
etc. Researchers observed that venture investors (e.g. angel funders and 
venture capitalists) prioritized entrepreneurial experience in their evaluation 
and selection of venture funding proposals (Zacharakis & Meyer, 2000). 
Studies also noted an increasing emphasis on knowledge and skills in other 
work environments outside of entrepreneurship. 

Consequently, numerous entrepreneurship studies acknowledged and 
built on Becker’s (1964, 1975) works on the role of human capital (defined 
primarily as education and job training) in entrepreneurial success. Findings 


showed that success in venturing is significantly higher for specific, task- 


related human capital than for general education (Unger, Rauch, Frese, & 
Rosenbusch, 2011). Specific human capital here refers to skills and 
knowledge acquired in previous new ventures or in the same industry as the 
new venture. Since many new ventures are founded by teams, the mix 
(heterogeneity of capabilities and synergy therein) of human capital present 
in the founding teams has also been found to be a significant predictor of 


startups’ fortunes (Colombo & Grilli, 2005). Recently, studies based on 


human capital have been complemented with studies of social capital and 


social networking (Baron, 2000; Davidsson & Honig, 2003). 


In connection with the above three streams of research, each with a long 
history, it is important to point out that the “mother” disciplines of 
economics, psychology, and sociology (Baum & Singh, 1994) dominated 
research in entrepreneurship. Home-grown theories of entrepreneurship 
remained rare during the first two decades of entrepreneurship research with 
the result that researchers in the “field” of entrepreneurship remained 
preoccupied with a perceived lack of legitimacy. Until the seminal article by 
Venkataraman (1997), there was considerable debate about whether 
entrepreneurship really is an academic domain in itself or not. At the same 
time, it was becoming increasingly clear that entrepreneurship was not just a 
buzzword, that there was a genuine phenomenon worthy of study by serious 
scholars. Attention was driven to the area by the global impact of 
entrepreneurs, in particular in new technology (Steve Jobs, Bill Gates, etc.), 
by student demand for entrepreneurship classes in universities (Katz, 2003), 
and by policy-maker interest in acknowledging entrepreneurship as a driver 
of jobs and economic development (Audretsch, Grilo, & Thurik, 2007). 
While historians such as Cantillon (1959 [c.1730]) and Braudel (1982) have 


for a long time noticed the role of entrepreneurs, business scholars, educators, 


policy-makers, and even economists only recently came to the consensus that 
entrepreneurship is an important phenomenon and a distinct domain worth 
studying on its own. It is interesting to note that this growing consensus 


developed in tandem with a rising tide of studies on entrepreneurial expertise. 


Studies of Entrepreneurial Expertise 


Ronald Mitchell’s dissertation (1994) was the first empirical investigation 
specifically on expertise in entrepreneurship. The studies in the dissertation 
focused on new venture formation as well as the composition and 
classification of expert information processing within this setting. Mitchell 
and colleagues (Mitchell, Smith, Seawright, & Morse, 2000) investigated 
expertise using a seven-country sample of entrepreneurs with differing levels 
of experience. The research used a script-scenario instrument that presented 
participants with paired response choices, of which one response represented 
script mastery and the other represented a socially desirable (distractor) cue 


(Mitchell et al., 2000, p. 982). For example, question 40 in the instrument 


offered two alternative completions to the starting phrase: “The new venture 
stories I recall ... (a) illustrate principles necessary for success or (b) are a 
telling commentary on the foibles of human nature which can rarely be 


predicted” (Mitchell, 1994, p. 79). (a) represents an expert understanding that 


prior experience builds principles for future success, while (b) exemplifies a 
distractor response indicative of a lack of expertise. 

The study used a panel of accomplished entrepreneurs to develop the 
script-scenario instrument. Thereafter, the degree of expertise in venture 
formation was inferred from participants’ responses to the script-scenarios in 
the instrument. Although the study used variation in simple experience to 
operationalize expertise, it was nonetheless the first systematic attempt to 
measure expertise in new venture formation. Furthermore, the results 


suggested important commonalities across country samples that indicated 


expert entrepreneurial scripts may be generalizable (Smith, Mitchell, & 
Mitchell, 2009, p. 821). 
Sarasvathy et al. (1998) subsequently used think-aloud protocols to 


study a representative sample of expert entrepreneurs, operationally defined 
in the established traditions of expertise research (Ericsson & Simon, 1993). 
In the study, four entrepreneurs and four bankers spent an hour thinking 
aloud as they worked through a set of five decision problems focused on risk 
and uncertainty. Results revealed that the bankers and entrepreneurs exhibited 
a “discernibly distinct cognitive approach for managing various types of risk” 


(Sarasvathy et al., 1998, p. 217). Entrepreneurs in the study accepted risks as 


irreducible, saw the worst-case scenario as a focal point for orienting their 
action, and worked toward influencing and controlling outcomes. Bankers 
picked a level of return (profit) they were comfortable with and then worked 
on managing risk. 

Another attempt to delineate pattern recognition capabilities in 
entrepreneurs came from Baron and Ensley (2006). The study used a 
retrospective interview technique with a participant pool consisting of 
entrepreneurs with differing levels of experience. Although, like Mitchell 
(1994), the study did not operationalize expertise according to the traditions 
of cognitive science, it nevertheless uncovered clear differences between 
more and less experienced entrepreneurs regarding their appraisal of business 
opportunities. The authors noted that the cognitive frameworks used by 
experienced entrepreneurs, “tended to focus on factors pertaining to financial 
success, rejecting ideas for new products or services that did not appear to 
offer manageable risk, the capacity to generate positive cash flow, and so on” 


(Baron & Ensley, 2006, p. 1340). By contrast, first-time entrepreneurs 


highlighted the novelty of the business idea, its competitive superiority, new 
technology, and their gut feel about the opportunity. 

Finally, a study by Unger and colleagues (Unger, Keith, Hilling, 
Gielnik, & Frese, 2009) of 90 South African business owners who founded 
and ran businesses investigated purposeful practice in entrepreneurship 
(Ericsson, 2016; Ericsson, Chapter 38, this volume; Ericsson, Krampe, & 
Tesch-Rémer, 1993). The authors defined the practice that they studied as 
“(Individualized self-regulated and effortful activities aimed at improving 
one’s current performance level” (p. 21). This practice is affected by 
education and cognitive ability, and is linked strongly to entrepreneurial 
knowledge (and both directly and indirectly to business growth). Drawing on 
work by Dunn and Shriner (1999) and Sonnentag and Kleine (2000) Unger et 
al. (2009) made four important suggestions: (1) the relevant practice activities 
may necessarily differ across domains (e.g. teaching, insurance, small 
business compared to sports, arts, and games); (2) ill-structured tasks or 
domains lead participants to practice a range of activities rather than 
repetitively practicing a few focal activities; (3) persistent, mindful 
engagement in activities for the sake of learning and improvement is the 
essential aspect of effective practice; and (4) though both are correlated with 
skill, current amount of practice activities is more correlated than that amount 
of accumulated past practice. 

A question that emerges from the abovementioned empirical work is: to 
what extent does experience at entrepreneurship lead to high performance or 
entrepreneurial success? The literatures on learning and human capital argue 
that new knowledge and skills are gained via experience (Kolb, 1984; Wang 


& Chugh, 2014), and that these are connected to performance (Becker, 1964; 


Cassar, 2006). However, empirical work on entrepreneurial human capital 


suggests that the overall experience—performance relationship is weak. In a 
meta-analysis that included 183 studies of small and medium businesses, 
Mayer-Haug and colleagues (Mayer-Haug, Read, Brinckmann, Dew, & 
Grichnik, 2013) found a mean correlation between experience and 
performance of less than 0.1. 

Yet even small “effects” can be practically significant (Unger et al., 
2011). Research at the firm level is suggestive of why this may be the case. 
Thompson’s (2010) review of the learning-by-doing literature in economics 
suggests that performance improvements otherwise attributed to learning are 
usually accompanied by other factors. For entrepreneurship this potentially 
suggests a hybrid model of learning that is more applied, action-oriented, and 
interaction-related. Adding to the learning debate, Frankish and colleagues 


(Frankish, Roberts, Coad, Spears, & Storey, 2013) investigated variables 


accounting for firm survival. They reject the learning-from-experience 
hypothesis. In their analysis, the lack of repetition opportunities (owing to 
task diversity) and the difficulty of interpreting the various causes of new 
venture survival, suggest that entrepreneurs improve performance only 
partially based on their experience at running new ventures. Toft-Kehler, 
Wennberg, and Kim (2014) also show this result for modest amounts of 
entrepreneurial experience. In their analysis, the performance impacts of 
entrepreneurial learning are only evident once entrepreneurs have gained 
extensive experience. Not until entrepreneurs were on to their third venture 
were performance differences attributable to learning observed. 

Such findings are consistent with the idea of passive learning. Ericsson 
(2004) pointed out that traditional models of learning suggest that when 
participants are introduced to a new activity they seek to attain an acceptable 


level of performance. As they gain initial experience, they make fewer 


mistakes and automate required skills. This results in less need to concentrate 
deliberatively in order to perform acceptably. Fifty hours or fewer represent 
enough practice to become socially competent in many recreational activities 
(for example, driving a car). However, once individuals achieve sufficient 
performance, improvement plateaus because they stop paying deliberative 
attention to the learning process. Thus, while the literature on entrepreneurial 
learning discussed above supports Ericsson’s observation, it does not answer 
the larger question he posed: “The fundamental theoretical challenge is to 
explain how most people and professionals reach a stable performance 
asymptote within a limited time period, whereas the expert performers are 
able to keep improving their performance for years and decades” (Ericsson, 
2004, p. S73). There exists a stream of research in entrepreneurship that 


tackles this question. We turn to that next. 


Summary of Effectuation Research 


Following cognitive science traditions of research into expertise, Sarasvathy 
(2001) used think-aloud protocols with a sample of 27 expert entrepreneurs to 
reveal their reasoning heuristics. The study was the first to directly draw upon 
Ericsson and Simon (1993). Participants in the study had, on average, 21.6 
years of entrepreneurship experience, had founded multiple new ventures 
(average 7.3 ventures started) and had taken at least one firm public 
(revenues were between $200 million and $6.5 billion). In other words, they 
had experienced everything entrepreneurship had to offer, and had 
demonstrated proficient performance in entrepreneurship. Industries and 
locations of the founded firms varied across the gamut of possibilities, 
ensuring the sample did not contain industry or geographic bias. 

The heuristics that were of interest in the study were termed 
“effectuation.” This concept came from Sarasvathy’s observation that while 
much of the managerial literature focused on causation, using historical data 
to predict outcomes in the environment, the expert entrepreneurs in her 
sample eschewed prediction, instead seeking to proactively “effect” changes 
into the environment combining their actions with those of their stakeholders 
(Sarasvathy, 2001). Participants were presented with a problem central to the 
domain of entrepreneurship: transform an idea into a new firm. The problem 
was broken into ten typical decisions in a 17-page protocol that each 
participant was asked to think through aloud. The instrument with the first 


two decisions in the problem is presented in the Appendix. 


While the cognitive capabilities underpinning expertise might be 
expected to be somewhat similar across domains, the purpose of studying a 
specific domain is to understand the distinctive heuristics and cognitive 
processes experts use in that domain (Chi, Glaser, & Rees, 1982). Analysis of 
the protocols led to the identification of the decision making heuristics 


(Sarasvathy, 2001), summarized in Table 22.1. Note that while these are a set 


of heuristics, they are bound together through an overarching logic that came 


to be called “effectual.” 


Table 22.1 Differences between predictive and effectual thinking 


Issue 


Definition 


Pilot-in-the- 


Predictive frame 


A process that takes a 
particular effect as 
given and focuses on 
selecting between 
means to create that 
effect. 


Accurate prediction is 


Effectual frame 


A process that takes a set of 
means as given and focuses 
on selecting between 
possible effects that can be 
created with that set of 
means. 


The future is shaped (at 


Plane both useful and least partially) by willful 
necessary. Control agents. Preference for 

View of the flows from the ability to working with factors 

future predict. perceived as directly 

controllable. 

Bird-in-the- Goals, constrained by Courses of action are based 

Hand limited means, on what is feasible with 
determine actions. means at hand. 

(Basis for taking 


action) 


Affordable loss Pursue risk-adjusted Pursue satisfactory 
maximum opportunity opportunities without 


(Predisposition —_and raise resources investing more resources 
toward risk and required to do so. Focus __ than stakeholders can afford 
resources) on upside potential. to lose. Focus is on limiting 


downside potential. 


Crazy Quilt Pursue stakeholders Stitch together partnerships 
needed to fulfill venture with customers, investors, 
(Attitude toward — goals and plans at etc. to shape the trajectory 
outsiders) minimum cost. of the new venture. 
Lemonade Contingencies are Contingencies are seen as 
principle obstacles to be avoided. opportunities for novelty 
creation, and hence to be 
(Attitude toward leveraged where possible. 
unexpected 
contingencies) 


In sum, the study found that with the accumulation of expertise, 
entrepreneurs develop a set of heuristics to deal with the uncertainty inherent 


in creating new ventures. According to Knight (1921), while risk consists of 


unknowns that are drawn from a known distribution, uncertainty has to do 
with unknown and even unknowable distributions. Therefore, entrepreneurial 
expertise consists of heuristics that minimize or eliminate reliance on 
prediction. Predictive strategies are more useful in dealing with risk. 
Effectual heuristics help deal with uncertainty. Expert entrepreneurs 
generally rejected heuristics based on prediction and forecasting. In the words 
of one of the experts: “I’ve always tended to be very skeptical about market 


research studies” (E14, Sarasvathy, 1998). Instead, their experience 


encouraged heuristics that sought to exert control over the environment. They 
shunned the notion of “placing bets” based on business plans, and in general 
challenged assumptions underlying predictive reasoning. In their view the 
future is an endogenous creation shaped by willful human beings. Hence it is 
not very useful to invest in predicting it. Unlike forms of expertise rooted in 
prediction, and typically associated with errors in judgment (Shanteau, 1992), 
effectuation internalizes “complex indeterminate causation” (Hoffman, Klein, 
& Miller, 2011) by incorporating co-creative human activity into the 
heuristics, to shape desired outcomes. 

The expert process for building new ventures emphasizes commitments 
from self-selected stakeholders driven by multiple motivations (in Table 22.1, 
the Crazy Quilt principle) as well as using unexpected contingencies as inputs 
along the way (the Lemonade principle). As a result, a new venture might 
develop in directions that are completely unforeseen at the time it was 
founded, e.g. it might start off in beer but end up in biotech — a path actually 
taken by Kiran Mazumdar-Shaw, founder of Biocon, and today one of India’s 
richest women. Indeed, it would have been impossible to predict in 1977 
when she quit her job in the beer industry, having been told she would not 


advance “because it’s a man’s work” (Krishnan, 2012) that she would go on 


to launch a startup that first produced enzymes and gradually expanded into 
pharmaceuticals. Thus, expert entrepreneurs learn that unpredictability grows 
out of their own actions as well as choices made by incoming stakeholders 
who negotiate and reshape the growing venture’s goals. Therefore, expert 
entrepreneurs have figured out that it is unwise to draft plans atop predictions 
and forecasts when they don’t know what business they are going to be in or 
which market they would enter or reshape or even co-create from scratch. 


In science, we are used to equating perceptions of control with the 


ability to predict things. However, we can already see that in the uncertain yet 
human domain of entrepreneurship, perceived control may be better served 
by relaxing the relationship between prediction and choice as much as 
possible. The control orientation in effectuation (the Pilot-in-the-Plane 
principle) is centrally concemed with preferring to work with factors directly 
within one’s control, and less with predicted factors outside one’s control 
(Sarasvathy, 2008). 

The above realization leads expert entrepreneurs to naturally work with 
means they already control as their starting point for action. Means under 
their control include their existing knowledge and networks as well as tastes, 
traits, and values (the Bird-in-the-Hand principle). Such inputs may or may 
not be connected with clearly defined end-goals. Specific goals emerge 
through the effectual process itself rather than from predetermined criteria for 
decisions. Complementing this means-driven heuristic is the fact that expert 
entrepreneurs are less interested in developing business plans aimed at raising 
outside resources than in working with people who want to work with them 
and using whatever resources these self-selected stakeholders bring with 
them. Finally, expert entrepreneurs operationalize control by focusing on the 
downside to reduce the worst-case scenario rather than placing a bet on the 
upside (the Affordable Loss principle). By letting their choices be guided by 
acceptable downsides, they exercise more perceived control over financial 
outcomes (Dew et al., 2009). Typically, this means small step investments 
and starting a new venture on minimal financial resources or what the trade 
press calls “bootstrapping.” 

In sum, effectuation consists in a set of control heuristics revealed from 
studying expert entrepreneurs. It is also important to point out that 


effectuation does not make any assumptions about personality characteristics 


of individuals or their motivations for starting a new venture. It merely 
presumes a modicum of individual initiative. Put differently, in contrast to 
more familiar strategies that offer necessary but insufficient criteria for good 
decisions, effectual reasoning offers sufficient yet unnecessary conditions for 
decisions and actions. 

Effectuation has gained strong interest within the entrepreneurship 
community, and dozens of articles have been published relating effectuation 
to a variety of topics (see Read, Sarasvathy, Dew, & Wiltbank, 2016b for a 
comprehensive review of over 200 articles). But despite its roots in the study 
of expertise and even while acknowledging the conceptual importance of it 


(Read & Sarasvathy, 2005), extant research into effectuation has not yet 


provided a comprehensive answer to Ericsson’s (2004) fundamental 
theoretical challenge, namely, why experts do not plateau. 

Before we advance toward the beginnings of an answer, a word about 
the domain specificity of entrepreneurship within the more general domain of 
management. While it may be easy to confound entrepreneurship within the 
broader scope of business in general, past research has found significant 
differences between managers and entrepreneurs (Busenitz & Barney, 1997). 
Perhaps owing to the uniquely uncertain situation in which entrepreneurs 


operate (Knight, 1921), our own research corroborates stark differences 


between managers and entrepreneurs. When we replicated the original study 
of expert entrepreneurs with expert managers, we found systematic 
differences between the two groups in the sort of expertise each accumulates 


within its domain (Read, Dew, Sarasvathy, Song, & Wiltbank, 2009). 


Purposeful Practice in Entrepreneurship 


Bridging the gap between acknowledging the importance of deliberate 
practice and actually showing its use by entrepreneurs requires the 
identification of constituent element(s) of deliberate practice in the 
entrepreneurial domain so it/they can be differentiated from (ordinary) 
experience. Although Unger et al. (2009) is the only study we are aware of 
that empirically investigated how entrepreneurs’ practice is related to 
achievement, other empirical works theorize about the connection. 
Importantly, the concept of effortful practice used in the Unger study draws 
on prior research on purposeful and deliberate practice in ill-structured 


domains, i.e. teaching, insurance agents, and software engineers (Dunn & 


Shriner, 1999; Sonnentag & Kleine, 2000; Sonnentag, Niessen, & Volmer, 
2006). These studies conceive of effective practice as being less constrained 
than the criteria for deliberate practice used in the foundational deliberate 
practice literature (Ericsson, 2016; Ericsson, Chapter 38, this volume; 
Ericsson et al., 1993). It is argued that in ill-structured domains, natural units 
of purposeful practice do not exist, standards of performance are fuzzy, and 
appropriate feedback is lacking. Therefore, Dunn and Shriner (1999) 
proposed a change to the concept of deliberate practice by eliminating some 
of the defining constraints: “At the foundation of the notion of deliberate 
practice ... is the fact that deliberate practice refers to activity that provides 
... Opportunity for learning and skill acquisition. It is possible that activities 
may look very different across domains yet serve this same purpose” 


(p. 633). Sonnentag and Kleine (2000) add that: “[DJeliberate practice 


activities performed within work contexts may differ from deliberate practice 
in other domains ... [T]/here are no specific types of activities per se that 
qualify for being deliberate practice” (p. 89, emphasis added). Instead, they 
recommended refocusing on two criteria: (1) the goal of competence 
improvement and (2) regularly executing an activity, which essentially 
encapsulate Ericsson and Pool’s (2016) concept of purposeful practice. 

Thus, for identifying high potential candidates for purposeful practice in 
entrepreneurship, we need to find specific practicable cognitive activities in 
which entrepreneurs engage. Prior work indicates that knowledge and skills 
developed in the new venture setting, or of direct relevance to the new 
venture, are more significantly connected with entrepreneurial performance 
than general work experience or general education (Unger et al., 2011). 
However, it is not clear: (a) how activities as diverse as those necessary to 
Start a venture map to improvements in specific knowledge structures that 
generate superior performance and (b) what kind of expertise develops from 
these activities. Thus, as intuitively appealing as these approaches are, 
significant questions remain regarding the kind of expertise (e.g. general 
problem-solving?) that develops from such activities and whether such 
activities actually contribute to improved entrepreneurial performance. 

An alternative to the strategy above is to take up Ericsson’s suggestion 
of focusing on representative activities that define the essence of a domain 
and can be mastered through deliberate practice (Ericsson, 2004). For 
example, in the domain of medicine Ericsson highlights that doctors may 
develop expert levels of performance in: (i) diagnosing test results; (ii) 
patient diagnosis; and (iii) surgical procedures. These are frequently 
occurring activities central to the domain that clinicians and surgeons practice 


frequently. Similarly, we can select activities every entrepreneur must 


perform to sustain a new venture. The goal is to identify activities that are 
central to the domain of entrepreneurship and that can be learned through 
repetition and feedback over the course of building new ventures. In order to 
identify such activities for research into entrepreneurial expertise, we need 


theoretical guidance. So we turn to that next. 


Toward Deliberate Practice in the 
Development of Entrepreneurial Expertise 


One reason for not embarking on empirical investigations of practice in 
entrepreneurship is the disagreement among entrepreneurship researchers 
regarding the degree to which type of practice may be important in the 


development of entrepreneurial expertise. According to Mitchell et al. (2007), 


“Mounting evidence in recent entrepreneurship literature suggests that the 
path to becoming an entrepreneur is not special, but is in fact general — rooted 
in the cognitive systems created by deliberate practice” (p. 14). Krueger 
(2007) makes a similar claim, and Smith et al. (2009) asserted that “It is now 
well accepted that entrepreneurial scripts are ... susceptible to, for example, 
deliberate practice-based change” (p. 821). 

However, in a critique of effectuation research as it stood in 2009, when 
only a small number of empirical papers on the topic had been published, 
Baron (2009) raised the important issue of practice in effectuation in 
particular and in entrepreneurship more generally, posing a key challenge: 
“In what tasks or activities do successful entrepreneurs demonstrate expert 
performance?” Given the importance of deliberate practice in the 
development of expertise in general, Baron’s critique is a legitimate and 
important one. Baron and Henry (2010) went on to argue that deliberate 
practice may not be possible in entrepreneurship, and that entrepreneurs 
instead either learn vicariously or (despite the domain-specificity hypothesis 
in expertise research) transfer skills learned via practice in other domains into 


their new ventures. This is in line with the notion of deliberate performance 


in Fadde and Klein (2010). Baron and Henry (2010) suggested several key 
activities amenable to on-the-job expertise acquisition such as: (1) 
recognizing and evaluating business opportunities; (2) building effective 
social networks; (3) acquiring essential resources; (4) making effective 
decisions; and (5) metacognition or self-regulation. 

Informed by the growing body of work on effectuation, we conducted a 
detailed examination of candidate activities amenable to purposeful practice 
that could then become the basis for deliberate practice in teaching 
entrepreneurship. We undertook a series of studies ranging in method from 
experience sampling to field experiments and in-depth qualitative interviews 
and case studies (Read, Sarasvathy, Dew, & Wiltbank, 2016a). Through these 


we found ourselves concurring with Yates and Tschirhart (2006) that it is 


extremely difficult to study and evaluate individuals’ overall skills in a 
domain, “But it is often feasible to isolate and examine how people deal with 
specific process elements” (p. 427). Therefore, decomposing — the 
entrepreneurial process into its constituent activities is a critical step in 
examining what aspects of performance might be conducive to deliberate 
practice (Shepherd, 2015). In our view the practice tasks in entrepreneurship 
may be more granular than Baron and Henry (2010) proposed (e.g. building 
effective social networks) and more homogeneous than Unger et al. (2009) 
proposed (e.g. they cover “a wide range of activities”). These aspects — 
granularity and homogeneity — are essential requirements that have been 
codified in the research on purposeful and deliberate practice (Ericsson, 
2016; Ericsson, Chapter 38, this volume; Ericsson et al., 1993). Purposeful 
practice must meet the following criteria: (i) activities must be decomposed 
or transformed into tasks that are meaningfully related to target performance; 


(ii) activities must be amenable to repeated practice; (iii) practice is 


motivated by a greater objective to improve performance; (iv) feedback on 
performance is available; (v) practice activities must take account of the 
performer’s current skill level; and (vi) activities must be within a “zone of 


proximal development” (Vygotsky, 1978), to be associated with the 


accumulation of expertise (Hoffman et al., 2013). To qualify as deliberate 
practice it is necessary to meet one additional criterion, namely that the 
practice is supervised and designed by a teacher, who has successfully trained 
other individuals to attain the target performance. 

In our search for such an activity conducive to purposeful and deliberate 
practice in entrepreneurship, we observed one in which entrepreneurs engage 
across all types of ventures, geographies, and times. We call this activity 
“The Ask.” In building a venture, entrepreneurs continually and iteratively 
interact with other people. Almost all of these interactions involve Asks. 
Asks can cover a variety of inputs necessary to creating a new venture that 
may include both intangibles (advice, introductions to network contacts) and 
tangibles (resources such as customer orders, supplier materials, labor, and 
money). While the “what” of The Ask differs across stages of the venture and 
particular stakeholders and situations, our observation is that the “how” of 
The Ask has repeatable common elements capable of continual practice and 
improvement. Likewise, while the “who” of The Ask differs (the identity of 
The Askee, from family, close friends, and network contacts to complete 
strangers), the activity of asking remains comparable. 

This led us to posit “The Ask” as one of the most important activities on 
which purposeful practice may be applied to improving entrepreneurship. 
Most importantly, asking is intrinsic to the early stages of the entrepreneurial 
process. Whether they like doing it or not, entrepreneurs have to engage in 


The Ask on multiple occasions each day in the course of launching a new 


venture. Hence repeated practice of The Ask is an inevitable feature of the 
startup environment (Ericsson & Smith, 1991). 

The Ask also fulfills the three other criteria identified from the literature 
on purposeful and deliberate practice. The Ask itself may not be inherently 
motivating for an entrepreneur but the larger objective is motivating, e.g. of 
successfully establishing a new venture. This provides the entrepreneur with 
powerful incentives for getting better at asking. Furthermore, an Ask 
typically creates spontaneous natural feedback for an entrepreneur, whether 
verbal or non-verbal, from The Askee or from surrounding observers. 
Immediate feedback in the form of rejection, acceptance (with the provision 
of new resources), or the introduction of new alternatives (perhaps an 
introduction to another person) provides the sort of feedback necessary for 
diagnosing failures and identifying improvement opportunities (Ericsson, 
2004, p. S77). Lastly, The Ask may be tailored to the skill level of the 
entrepreneur. Indeed, we can precisely conceptualize a natural progression 
from apprentice to higher proficiency levels of asking. 

An Ask familiar to anyone who has experienced entrepreneurship is the 
investor “pitch.” Pitching is well known from business plan competitions as 
well as TV shows such as Shark Tank (in the US) and Dragon’s Den (in the 
UK). The process of obtaining resources from investors is widely perceived 
to be quite central to the entrepreneurial process for at least three reasons: (1) 
entrepreneurs may believe they do not have sufficient personal resources to 
fund their new venture; (2) entrepreneurs may not want to fund a firm with 
their own money; or (3) entrepreneurs may need outside funding to grow a 
firm quickly. The pitch has a particular recipe in that the entrepreneur is 
usually targeting either angel investors or venture capitalists (who) with a 


defined request for resources (what) based on an opportunity the entrepreneur 


envisions (why). The pitch is carefully crafted to target investors who can 
best provide the required resources, and entrepreneurs may shuttle from one 
prospective investor to another pitching and pestering until they either get 
what they are seeking, get turned away, or get a “maybe later” response 


(Hellman, 2007). The sales pitch is another example of the same generic type 


of Ask, targeted at potential customers instead of investors. 

It is our observation that the nomenclature of “the pitch” hides at its 
heart a causal (or predictive) as opposed to an effectual (or co-creative) Ask. 
The pitch consists of an Ask where The Asker knows exactly what he or she 
wants and from whom and an estimate of the upside to induce The Askee to 
invest. It is not difficult to imagine less targeted and less goal-driven Asks 
(i.e. more effectual than the pitch). In other words, the pitch is only one type 
of Ask. In the daily practice of entrepreneurship, we observe four different 


types of Asks: 


e The first is familiar to people as “the pitch.” In its most simple form, 
it consists of an entrepreneur predicting who has what he or she wants 
and targeting particular stakeholders and saying “Please” (... give me 
X...). 


e The second is the Transactional Ask. This is also a causal Ask in 
which an entrepreneur targets particular stakeholders but this one 
includes negotiation on a quid pro quo basis. The heart of this Ask is 
“You give me X and I’ll give you Y,” with Xs and Ys that can vary 
through interaction between The Asker and The Askee. 


e A third type of Ask is what we describe as a Tentative Ask. This is an 
Ask that is low on both prediction and negotiation. It asks “Might you 


B] 


be willing to ...?’ 


e Lastly, there is a Co-creative or effectual Ask, which can be 
exemplified by “What would it take for you to ...?” This Ask is the 
most open-ended because it allows stakeholders to set their own terms 
(i.e. “This is what I would need ...”) without the entrepreneur having 
to predict them in advance. It also allows stakeholders to help shape 
the venture in return for their commitment to become involved in 


some way. Stakeholders thus have a hand in co-creating the new firm. 


These examples indicate the breadth of possibilities for purposeful practice 
based on The Ask. For example, novice entrepreneurs can start with tentative 
Asks and practice their way to co-creational relationships involving multiple 
complex intersubjective Asks over time that are the hallmark of 
entrepreneurial expertise. 

We have field-tested The Ask as a potential candidate for purposeful 
practice in entrepreneurship. Although the empirical studies of The Ask are 
currently in progress, we propose that Asks are indeed a major building block 
in the development of expertise in entrepreneurship. For the purposes of the 
rest of this chapter, however, we focus on the implications of The Ask that go 


beyond the field of entrepreneurship. 


Implications of The Ask: Purposeful Practice 
in Domains Characterized by Complex 
Indeterminate Causation 


As mentioned earlier, one of the distinguishing characteristics of 
entrepreneurship as a domain of expertise consists in its having to deal with 
uncertainty (Knight, 1921; Alvarez & Barney, 2007; Foss & Klein, 2012). 


That suggests an interesting connection with the concept of “complex 


indeterminate causation” (Hoffman et al., 2011). As presented by the authors, 
complex indeterminate causation (CIC) represents a situation in which 
rational individual decision making is hindered by an inability to discern a 
clear chain of causality. “In real-world settings, the evidence for causation is 


typically too ambiguous to permit valid (i.e., deductive) reasoning, so 


[rationality] is not a generally useful standard” (Hoffman et al., 2011, p. 419). 
Among other things, CIC offers an explanation for why some domains lend 
themselves to the accumulation of expertise that has been studied in much of 
the existing literature. Termed “Type 1 domains,” environments such as the 
game of chess, where it is relatively easy to establish a causal connection 
between proficiency at specific activities that can be practiced and 
performance within the domain, have provided the foundation for much of 


the study of expertise. In contrast: 


Type 2 domains are ones in which the ostensive principal task goals 
involve the prediction of individual or aggregate human activity. Human 
activity fails to provide the needed cues for timely feedback, which is 


one of the reasons why it can be difficult to achieve expertise. At a 


collective level, human activity is subject to too many unpredictable 


events and decisions. 


(Hoffman et al., 2011, p. 405) 


Expertise researchers sometimes identify Type 2 domains as 
problematic, and as one source of errors and biases that have been observed 
in the judgments of experts (Shanteau, 1992). But because it is precisely 
these complex and uncertain situations which are the domain of the 
entrepreneur, we suggest a reconsideration of expertise that is specific to CIC 
and Type 2 domains. From our work on effectuation, one overarching finding 
is the rejection of prediction in the decision making heuristics of expert 
entrepreneurs. Our findings indicate that causality, or even the search for 
causality through prediction, is rejected by our expert entrepreneur sample. 
Instead it is replaced with alternative heuristics centered on control that 
seems to emerge as entrepreneurs accumulate expertise. Simply put, instead 
of trying to predict the future, the expert entrepreneur subjects in our studies 
dealt with causality by endogenizing it. By taking the actions they can, with 
the resources they had available, at risk levels they found acceptable, and 
with partners who are committed to working with them, they view their own 
actions as setting off a causal chain of events over the outcomes of which 
they have some influence. 

Such a view of the world inverts CIC from a liability (if you are 
counting on prediction to help establish causality) to an asset (if it is you who 
is creating both CIC and the desirable outcomes it engenders). 
Entrepreneurship offers a domain of expertise that is perfectly suited to 


viewing the world this way. In entrepreneurship we observe active agents 


actually initiating the events and decisions that generated the effects 
described by Hoffman et al. (2011, p. 406): 


What counts as an “effect” is predicated upon events and decisions that 
are themselves influenced by forces and abstractions. They in turn give 
rise to new (or continuing) events and decisions that are in turn 


embodied in new (or continuing) forces and abstractions. 


To the extent that CIC settings involve human actions and interactions, 
we believe a closer and deeper examination of stakeholder relationships 
would be key to developing expertise in Type 2 domains. Whether The Ask 
itself would be the appropriate practice task in all of these domains would be 
an interesting empirical question. More generally, this insight opens up an 
interesting philosophical basis for expertise studies involving CIC, namely 


the role of intersubjectivity (Mead, 1927) and design (Simon, 1969). 


Entrepreneurship is inherently co-creative, requiring not just interaction 
within teams but also engagement with others outside the firm in order to be 
successful. As one example, the specific activity of asking ranges from 
simple types of interaction between two people to complex intersubjective 
interactions in which learning cannot be cleanly separated into the individual 
cognition of each person involved. This suggests the need to examine 
expertise acquisition from the perspective of situated and social cognition as 
well as from the point of view of individual cognition (Cornelissen & Clarke, 
2010; Mieg, 2006; Mitchell, Randolph-Seng, & Mitchell, 2011). 

Of course, the notion that the appropriate practice task (and therefore 
expertise acquisition) might involve teams and constellations of individuals 
does not uniquely emerge from or relate to entrepreneurship (Taylor & 
Thorpe, 2004). Acknowledging this, Hoffman, Feltovich, and Ford (1997) 


urged researchers to consider the “expert-in-context” as the minimum unit of 
analysis. The minimum unit of analysis for expert performance in a 
fundamentally interactive context is one that intrinsically involves 
information exchange, decisions, and actions by other people that affect the 
performance of the activity. For instance, when activities incorporate 
disclosures of information from another party that may facilitate or hinder 
performance at an activity, the performer’s outcomes are contingent on the 
other party (Heide & Miner, 1992). 

We already have insights regarding the value that may be added by 
incorporating ideas and concepts from situated and social cognition into 
expertise studies. Mieg (2006) provided the macro-social view on expertise, 
emphasizing that a “social conception of the expert differs from other ones 
discussed ... such as the expert as an outstanding individual nominated by 
peers ... and the expert defined by his/her superior performance” (p. 743). 
Instead, Mieg explained the problematic nature of defining expertise from a 
broader social perspective where almost anyone can be an expert under the 
right circumstances, and where “expert” may simply mean that you are 
regarded as one by others, particularly within a profession (Abbott, 1988). An 
important take-away from Mieg’s analysis is the socially defined nature of 
performance criteria for expertise. Here, the social impinges on expertise 
primarily from without by governing standards for activity performance as 
well as structuring the broader context in which activities are valued. 

But the social also impinges on activities from within, i.e. when the 
thoughts and actions of others affect the performance of an activity by an 
individual. Continuing with our focus on The Ask, social psychologists have 
made important discoveries about how individuals’ help-seeking behavior is 


affected by their perceptions of others’ willingness to help (Flynn & Lake, 


2008). It turns out that these perceptions significantly affect an individual’s 
performance in the activity of asking in a strongly negative way, i.e. help- 
seekers underestimate the likelihood that others will help them by as much as 
50 percent. Non-expert askers appear to systematically believe that others 
will say no, even when they have no hard evidence on which to base such an 
assumption. Expert entrepreneurs, however, achieve better calibration 
through the actual practice of asking. Furthermore, extant evidence suggests 
that for the critical activity of asking in entrepreneurship, the mutuality of the 
situation can have a strong influence on activity performance. In other words, 
just the fact that other people are involved changes the nature of the activity 
in important ways. 

Compare, for example, the appropriate practice tasks in industrial 
process control with those in law. A process control analyst may practice 
observation and monitoring. The focus is on the process, which can be 
objectively measured and refined. By contrast, the representative activities 
that define the essence of the legal domain (e.g. client preparation, in-court 
performance) are likely to intrinsically involve lawyerly interactions with 
other people in which those people (e.g. clients, judges, or juries) take an 
active role rather than a passive one. The activities to be worked on are 


fundamentally interactive. 


e We have studies of taxi drivers’ expertise (e.g. memorization 
capabilities; Kalakoski & Saariluoma, 2001) and expert salespersons 
(Ko & Dennis, 2004) so why not studies of expert headhunters (since 
the crucial activity of hiring is also fundamentally interactive) or 


salespeople selling? 


e We also have studies of designers’ and inventors’ expertise 


acquisition (Cross, 2004; Mieg, Bedenk, Braun, & Neyer, 2012) so 


why not studies of innovators’ activities in realizing a useful and/or 
commercial application for such inventions (e.g. in which success 
depends on interaction with partners, clients, and providers of 
complementary goods/services)? This “why not” may add depth to the 
context of entrepreneurship we consider in our research as well as 
shedding light on the perhaps surprising finding that many inventors 
do not wish to become entrepreneurs (Kassicieh, Radosevich, & 
Banbury, 1997). 


e We have studies of expert firefighters (Klein, 2009) so why not 
studies of master marriage counselors (e.g. where performance as a 
mediator intrinsically depends on the engagement and co-participation 


of parties to the dispute)? 


The common attribute to all of our “why nots” is that each of these 


domains involves individual or aggregate human action and interaction, an 


area of emerging interest to expertise scholars (Hoffman et al., 2011). In sum, 
investigations along these lines offer several research avenues, as follows. 


First, human beings are by nature social and embodied (Joas, 1996). As 


a consequence, we must consider loading sociality into the conceptual 
structure of the activities that experts become very good at, in addition to 
considerations of the individual actor. This will require an emphasis on 
mutuality in the situation, i.e. the influence of others on the immediate 
activity environment. 

Second, because activities such as The Ask are_ inherently 


intersubjective, one way of framing them is as joint problems which different 


parties are looking for solutions to. Research suggests that the construction 
and maintenance of a shared conception of a problem is a fundamental 
characteristic of mutual learning in these situations (Roschelle & Teasley, 
1995). Shared conceptions may be developed tentatively via dialogue on an 
ad hoc, on-the-fly basis. Such a perspective builds well atop current work in 
team cognition (Hoffman et al., 2013), expanding the canvas to include 
looser constellations of more diverse stakeholders jointly engaged in the 
activity of co-creation (Aarikka-Stenroos & Jaakkola, 2012). Such a 
perspective might open questions regarding the exchange and spillover of 
fine-grained information among individuals, both deliberate and accidental. 
In situations characterized by CIC, it is difficult to determine what 
information is salient, what might be useful, and what should be held as 
proprietary. For example, in entrepreneurship, asking a user to try out your 
service might involve learning about the problems they experience, their 
goals and plans, e.g. asking them to share proprietary information. There may 
be correction of errors and misperceptions. In this example, a degree of open 
information sharing reduces the information asymmetry that otherwise exists 
between the entrepreneur and potential users. Determining what constitutes 
purposeful and deliberate practice and expertise in such tasks may prove a 
useful research direction. 

Lastly, once we embrace situations involving CIC and seek to explicitly 
incorporate individual or intersubjective human activity into expertise 
models, we will need to consider a multiplicity of motivations and intentions 


(Ainslie, 2001; March, 1978). As with the introduction of sociality, shared 


conceptions, and information exchange, multiple motivations and intentions 
introduce dynamism into the activity space. It is possible that expertise in 


such activities may incorporate skills such as persuasion and negotiation, 


which are inherently intersubjective. There may also be connections to 
contemporary work in expertise research on individual preferences for 
explanations of complex events or those with indeterminate causality (Klein, 
Rasmussen, Lin, Hoffman, & Case, 2014). 


Agenda for Future Research 


In this section, we outline three empirical designs that seek to incorporate the 
intersubjective into expertise research. The first applies the method of 
experience sampling to a hypothetical design of our own creation. The 
second adapts an existing design from psychology and the third represents a 
more speculative possible design. We cast these examples as specific to 
investigations of entrepreneurial expertise and the superior performance of 
asking. But each of these also represents ways to investigate situations 
characterized by CIC and can therefore be extended to studies of expertise in 


domains other than entrepreneurship. 


Experience Sampling 


Our first design uses an experience sampling methodology (ESM) (Flugel, 
1925; Csikszentmihalyi, Larson, & Prescott, 1977). The goal of the study 


would be to track The Asks of small business owners over time and then use 


content analyses to examine the data (Ericsson & Simon, 1993). With such an 


approach, we could compare data both within participants (looking for 
sequences that represent the impact of experience and purposeful practice) 
and across participants (looking for differences that might stem from 
expertise, situational factors, or personal factors). More generally, experience 
sampling seems well suited to work that seeks to understand the nature of 
expertise in situations characterized by CIC. Scollon, Prieto, and Diener 


(2009, p. 8) articulate the virtues of the method as: 


First, ESM allows researchers to better understand the contingencies of 
behavior. Second, ESM takes psychology out of the laboratory and into 
real-life situations, thus increasing its ecological validity. Third, ESM 
allows for the investigation of within-person processes. Fourth, 
researchers can avoid some of the pitfalls associated with traditional 
self-reports, such as memory biases and the use of global heuristics. 
Fifth, ESM answers the call for the greater use of multiple methods to 
study psychological phenomena. 


Investigations specific to expertise that is developed intersubjectively 
could consider designs that sample both sides of the interaction, using 
powerful approaches like protocol analysis to draw insights from the 


significant quantity of data such a design might yield. 


Helpfulness Experiment 


This design is a modified replication of a study within the domain of 
entrepreneurship. Interestingly, the study already uses an Ask task. Flynn and 
Lake (2008) showed that people underestimate the helpfulness of others and 
overestimate their ability to help. In a series of studies where subjects had to 
ask strangers for a variety of favors (completing a questionnaire, borrowing a 
cellphone, getting a tour of the gym, etc.) the authors found their subjects 
systematically underestimated the likelihood (sometimes up to 50 percent) 
that a stranger would respond positively to their request. This study could 
easily be replicated with entrepreneurs. 

Our proposed adaptation of the study would record the entrepreneurs in 
the actual process of asking. We would pair entrepreneurs to engage in the 
activity of asking each other for help regarding their venture. We would then 
follow Flynn and Lake (2008) in analyzing what they asked for, if they got 
something useful, and how they phrased the question. With these and 
additional data we would collect from recordings or from dyadic asking 
interactions, we could isolate the mechanisms that inhibit and promote asking 
behavior in entrepreneurship. 

Such an approach would lend itself to manipulations both of the focal 
Ask and also of the subject entrepreneurs. We know that entrepreneurs vary 
on numerous important dimensions such as economic context (Boyd & 


Vozikis, 1994), or level of (over)confidence (Koellinger, Minniti, & Schade, 


2007). The use of experimental methods may allow us to delineate specific 


relationships between variables of interest to behavioral economists and 


social psychologists with outcomes of interest to entrepreneurship researchers 


and cognitive scientists focused on the development of expertise. 


Sequencing 


Seeking to understand the heuristics of individuals engaged in The Ask, we 
imagine developing a deck of cards, each of which represents a possible 
action in an Ask sequence (Frese et al., 2007). The content of these cards 
would necessarily be derived from observing actual entrepreneurs engaged in 
the practice of The Ask, perhaps a by-product of one of the first two studies. 
The cards could be used in a variety of ways. After identifying a 
representative sample of entrepreneurs, we could create a study where we 
compare the planned sequence of actions the entrepreneur prepared before an 
Ask against the actual sequence used during The Ask. Or we could contrast 
sequences prepared by experts with sequences prepared by novices. Selection 
(or non-selection) of particular actions could be as important as the sequence 
itself. Additionally, such an instrument offers significant teaching 
possibilities as well. In domains where the actions are contingent and 
intersubjective, turning the cards into a game where people play in dyads and 
use the cards instead of communicating might yield advancements in 
understanding which patterns emerge from practice. This would be in line 
with recent work on “ShadowBox” training designs in which apprentices 


compare themselves with experts (Klein & Borders, 2016). 


Conclusion 


We live in a time marked by accelerating technology, interconnectedness, and 
complexity. Entrepreneurship offers one of the most promising domains for 
research that embodies those characteristics. In fact, extant studies into 
entrepreneurial expertise have already underscored the need to expand our 
understanding of expertise to that of the intersubjective expert. Fortunately, 
recent work on expertise provides a foundation to take on these ambitious 
challenges, and points to some of the rewards we can expect from 
undertaking those challenges. We look forward to intersubjectively 
addressing some of these challenges not only with our colleagues in 
entrepreneurship research and in research into expertise, more generally, but 
also with others who would like to self-select into this co-creative enterprise. 
In addition, the scientific study of entrepreneurial expertise illustrates the 
prospects for taking the broader field of expertise studies into new areas and 
domains that are interestingly and significantly different from the domains 


that have been “classically” studied. 


Appendix 
The Venturing Instrument 


Introduction 


In the following experiment, you will solve two decision problems. These 
problems arise in the context of building a new company for an imaginary 
product. A detailed description of the product follows this introduction. 

Although the product is imaginary, it is technically feasible and 
financially viable. The data for the problems have been obtained through 
realistic market research — the kind of market research used in developing a 
real world business plan. 

Before you start on the product description and the problems, I do need 
one act of creative imagination on your part. I request you to put yourself in 
the role of an entrepreneur building a company — i.e. you have a little money 
of your own to start this company, and whatever experience you have to date. 

Throughout the experiment you should talk aloud the thoughts you 


are having. Please start by reading aloud the following instructions. 


Description of the Product 


You have created a computer game of entrepreneurship. You believe you can 
combine this game with some educational material and profiles of successful 
entrepreneurs to make an excellent teaching tool for entrepreneurship. Your 
inspiration for the product came from several reports in the newspapers and 
magazines about increasing demand for entrepreneurship education; and the 
fact that a curriculum involving entrepreneurship even at the junior high or 
high school level induces students to learn not only business-related topics 
but math and science and communication skills as well. 

The game part of the product consists of a simulated environment for 
starting and running a company. There are separate sub-simulations of 
markets, competitors, regulators, macroeconomic factors, and a random 
factor for “luck.” The game has a sophisticated multimedia interface — for 
example, a 3D office where phones ring with messages from the market, a 
TV that will provide macroeconomic information when switched on, and 
simulated managerial staff with whom the player (CEO) can consult in 
making decisions. At the beginning of the game, the player can choose from a 
variety of businesses the type of business he/she wants to start (for example: 
manufacturing, personal services, software, etc.) and has to make decisions 
such as which market segment to sell to, how many people to hire, what type 
of financing to go for, etc. During the game, the player has to make 
production decisions such as how much to produce, whether to build new 
warehouses or negotiate with trucking companies, etc.; marketing decisions 


such as which channels of distribution to use, which media to advertise in and 


so on; Management decisions involving hiring, training, promoting, and firing 
of employees, and so on. There is an accounting subroutine that tracks and 
computes the implications of the various decisions for the bottom line. The 
simulation’s responses to the player’s decisions permit a range of possible 
final outcomes — from bankruptcy to a “hockey stick.” 

You have taken all possible precautions regarding intellectual property. 
The name of your company is Entrepreneurship, Inc. The name of the 


product is Venturing. 


Problem 1: Identifying the Market 


Before we look at some market research data, please answer the following 
questions — one at a time. (Please continue thinking aloud as you arrive at 


your decisions. ) 
(1) Who could be your potential customers for this product? 
(2) Who could be your potential competitors for this product? 


(3) What information would you seek about potential customers and 


competitors — list questions you would want answered. 


(4) How will you find out this information — what kind of market 


research would you do? 


(5) What do you think are the growth possibilities for this company? 


Problem 2: Defining the Market 


In this problem you have to make some marketing decisions. Based on 


secondary market research (published sources, etc.), you estimate that there 


are three major segments who are interested in the product: 


Segment Estimated total size 


Young adults between the ages of 15 and25 20 million 


Adults over 25 who are curious about 30 million 
entrepreneurship 
Educators 200,000 institutions 


The estimated dollar value of the instructional technology market is $1.7 
billion. 

The estimated dollar value of the interactive simulation game market is 
$800 million. 

Both are expected to grow at a minimum rate of 20 percent p.a. for the 


next five years. 


The following are the results of the primary (direct) market research 
that you have completed 


Survey #1: Internet users were allowed to download a scaled down version 
(game stops after 15 minutes of playing) of the prototype and were asked to 
fill out a questionnaire 


You get 600 hits per day; 300 actually download the product. You have 500 


filled out questionnaires. 


Willing to pay Young adults 
(S) (%) Adults (%) Educators (%) 


50-100 45 26 D2 
100-150 32 38 30 
150-200 15 22 16 
200-250 8 a 2 
250-300 0 rs) 0 
Total 100 100 100 


Survey #2: The prototype was demonstrated at 2 Barnes and Noble and 3 


Borders bookstores 


Willing to pay Young adults 


($) (%) Adults (%) Educators (%) 
50-100 51 21 65 
100-150 42 49 18 
150-200 7 19 10 
200-250 0 8 7 
250-300 0 3 0 
Total 100 100 100 


Survey #3: Focus group of educators (high school and community college 
teachers and administrators) 


The educators who participated in the focus group find the product exciting 


and useful — but want several additions and modifications made before they 


would be willing to pay a price of over $150 for it. As it is, they would be 
willing to pay $50—80 and would demand a discount on that for site licenses 
or bulk orders. 

Both at the bookstore demo and the focus group, participants are very 
positive and enthusiastic about the product. They provide you good feedback 
on specific features and also extend suggestions for improvement. But the 
educators are particularly keen on going beyond the “game” aspect; they 
make it clear that much more development and support would be required in 
trying to market the product to them. They also indicate that there are non- 
profit foundations and other funding sources interested in entrepreneurship 
that might be willing to promote the product and fund its purchase by 


educational institutions. 


Based on your market research, you arrive at the following cost 
estimates for marketing your product 


Internet $20,000 upfront + $500 per month 
thereafter 
Retailers $500,000 to $1 m upfront and support 


services and follow-up thereafter 


Mail order catalogs Relatively cheap — but ads and demos could 
cost $50,000 upfront 
Direct selling to schools Involves recruiting and training sales 


representatives except locally 


Competition 


None of the following four possible competitors combine a simulation game 


with substantial education materials — you are unique in this respect. 


Company 


Maxis 


Microprose 


Sierra On- 
Line 


Future 
Endeavors 


Product 


Sim City 


Civilization 


Caesar 


Scholastic 
Treetop 


Description 


Urban planning 


simulation 


Civilization 
building 
simulation 


City building 


simulation 


CD-ROMs of 
scholastic books 


Price 
per 
unit 


29.95 


50.00 


99.00 


n/a 


Sales ($) 


30m 


20m 


18m 


1 m (New Co. < 
1 yr. old) 


The game companies are making a net return of 25 percent on sales. 


At this point, please take your time and make the following decisions: 


(Please continue thinking aloud as you arrive at your decisions) 


1. Which market segment/segments will you sell your product to? 


2. How will you price your product? 


3. How will you sell to your selected market segment/segments? 
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Introduction 


Writing extended texts for publication is a major cognitive challenge, even 
for professionals who compose for a living. Serious writing is at once a 
thinking task, a language task, and a memory task. All three cognitive 
functions can be overloaded by the demands of composing an extended text 
that clearly communicates the author’s thoughts and artfully connects with its 
intended readers. The material that was presented in the chapter on 
professional writing in the first edition of this handbook is still current and 
useful. The present revised chapter in part updates the points made previously 
with new findings that appeared in the literature since the first edition. In 
addition, the section on skill acquisition updates and focuses on an advanced 
stage of development — knowledge crafting — attained only by expert, 
professional writers. Finally, the revised chapter also provides a new section 
on how neuroimaging of the brain has shed light on the cognitive processes 
involved in written composition. 

This chapter will open with the question of how to define the concept of 
professional writing, and an explanation of the demands that writing 
processes make on cognitive resources. Neuroimaging of the writer’s brain 
during composition processes will then be described. Next, the characteristics 
of professional-level writers and writing expertise will be enumerated and 
explored. In the final section, the acquisition of writing skill will be 


discussed. 


Defining Professional Writing 


Defining expertise in writing is difficult because the task is ill-structured 


(Simon, 1973) and because the types of texts generated by professionals are 


so varied. An expert in chess successfully checkmates the opponent and the 
allowable moves in the game are defined clearly. By contrast, the writer’s 
task is poorly structured with multiple goals that are described in very general 
terms (e.g. the text must be coherent) and the means to these ends are not 
well specified and agreed upon. Further, the domains of writing are extensive 
and non-overlapping. Professional communication is a term that readily 
encompasses the writing done in the workplace, such as “reports, proposals, 


presentations, multimedia, [and] Web pages” (Schriver, 2012, p. 276). These 


professionals often devote considerable time and effort to writing, even 
though they define their job in other ways (e.g. professors, scientists, 
engineers, business managers, government bureaucrats, and diplomats). 
However, those who write for a living also include journalists, novelists, 
screen writers, poets, technical writers, and authors of everything from 
scientific tomes to cook books. The jobs of career writers who compose on 
technical and professional subjects for a living certainly differ from those of 


engineers and administrators in an organization, for example (Couture, 1992). 


The literature on writing expertise samples all these domains and, 
despite the variability, the common elements offer lessons with wide 
applicability. The goal of the present chapter is to summarize the relevant 
psychological evidence applicable to all professional writers rather than 
exploring in detail a single form. The chapter should be of interest, then, to 


professionals who write, writing professionals in business, industry, and the 


non-profit sectors as well as to novelists, playwrights, poets, journalists, and 
non-fiction authors who make a living primarily, if not solely, through their 


writing. 


Cognitive Demands of Writing 


In a seminal psychological model of written composition, Hayes and Flower 
(1980) distinguished three basic processes of text production: planning ideas, 
translating ideas into text, and reviewing ideas and text. As will be discussed, 
these are not linear phases of text production, starting with planning and 
ending with reviewing. Instead each process can be and often is invoked 
throughout all phases of text development, from pre-writing to a final draft. 
In subsequent updates to the model, Hayes (1996, 2012) stressed how the 
limitations of working memory constrain these basic processes. He further 
stressed the role of motivation and affect in composition. As will be seen 
later in the chapter, professional writers owe their success in part to learning 
how to overcome working memory constraints and to managing the 
frustrations and other negative emotions that can block less experienced 


writers. 


Fundamental Writing Processes 


Planning includes generating concepts, organizing them, and setting goals to 
be achieved in the structure of the text. The writer must struggle concurrently 
with two separate problems (Bereiter & Scardamalia, 1987). On the one 
hand, there is the problem of what to say. The writer mentally represents and 
manipulates beliefs and facts about the topic within the content “problem 
space.” On the other hand, there is the different problem of how to say it. In 
the rhetorical “problem space,” the writer finds ways to achieve the goals of 


the composition. 


The products of planning fall along a continuum ranging from non- 
verbal imagery to abstract propositions and word images that are more 
readily translated to written text (Flower & Hayes, 1984). Sequences of word 
images or mental pre-texts are what Witte (1987) called “the last cheap gas” 
before the major cost of concretizing plans into written text. The only visible 
products of a writer’s planning consist of diagrams, outlines, lists, and other 
“notebooks of the mind,” to borrow John-Steiner’s (1985) description. These 
extemnalized plans are typically cryptic and intended only for the writer’s 
private use. 

The translation of ideas into sentences and paragraphs yields a draft of a 
text that the author intends to eventually be read and understood by others. 
Several language processes, largely non-conscious, operate on images and 
propositions to generate a sentence, such as selecting words and assigning 
them to syntactic roles (Bock & Levelt, 1994). Besides translating ideas into 
a sentence, writers must also generate cohesive links among sentences, 
establishing coherence at both local and global levels in the text (McCutchen, 
1984). For written output, graphemic representations must be specified for 
spelling each word (Caramazza, 1991). The motor transcription of words into 
written characters often closely follows sentence generation in time and is 
difficult to distinguish through natural observation alone. It can, therefore, be 
viewed as a sub-process of translating (Berninger & Swanson, 1994). 

Reviewing the text involves reading and editing operations that detect 
faults at multiple levels of text structure, ranging from local matters of 
diction, spelling, or punctuation to the coherence of the whole text. It further 
involves editing ideas and other products of planning (e.g. lists, outlines, 
diagrams) prior to their conversion to sentences and extended text. The 


reading and editing processes are complex because so many things can go 


wrong in a plan or a text at so many different levels of structure (Hayes, 
Flower, Schriver, Stratman, & Carey, 1987). At the word level of a text, for 
example, the writer may detect problems with the graphemes used for 
spelling or with the shade of meaning conveyed. At the sentence level, 
grammatical errors, semantic ambiguities, alternative interpretations, and 
problems of reference must be evaluated. The writer must further address 
faulty logic, errors of fact, and other inconsistencies with world knowledge, 
on the one hand, and problems with text structure, incoherence, 
disorganization, on the other. As if this were not enough, the writer must also 
consider the needs of the audience, looking for the right tone and degree of 
complexity. 

As noted earlier, the basic processes of writing can be distinguished 
from the temporal phases of composing a text. The research literature clearly 
shows that planning, translating, and reviewing do not occur in a linear 
sequence in text production; rather, they occur and reoccur in complex 
patterns interwoven through pre-writing, first draft, and subsequent draft or 
revision phases of composition (Kellogg, 1994). The interactions among 
planning, translating, and reviewing are responsible for the development of 
sophisticated descriptions, narratives, and arguments. During a pre-writing 
phase, for example, the writer may plan, review the ideas and mental pre-text, 
and then plan some more to produce, say, an outline of topics to be included 
in a first draft. Although pre-writing can be brief, experts approaching a 
serious writing assignment may spend hours, days, or weeks thinking about 
the task before initiating a draft. 

During the first draft phase, writers continue to plan, generate sentences, 
read the developing text, and edit ideas and text. Although some writers try to 


produce a perfect first draft that requires only minor corrections, the first 


draft is typically revised through one or sometimes many subsequent drafts. 
Fitzgerald (1987) defined the revision phase of reworking the first draft as 
“identifying discrepancies between intended and instantiated text, deciding 
what could or should be changed in the text and how to make the desired 
changes, and ... making the desired changes” (p. 484). In revision, as in the 
original drafting of a text, the writer may engage again in planning and 


translating as well as further reviewing. 


Supporting Brain Regions 


Neuroimaging methods such as functional magnetic resonance imaging 
(f{MRI) can highlight highly localized brain regions that mediate specific 
cognitive processes. Depending on the temporal resolution of the scanner, the 
neuroimages may reflect a summation of multiple cognitive activities that all 
took place within a window of several seconds. This is especially true with 
positron emission tomography (PET). Another limitation is that {MRI and, to 
a lesser extent, PET, restrict the writer’s head and upper limb movements, 
restricting, too, the scope of studies that may be undertaken. Despite these 
limitations, progress is being made in identifying brain regions that support 
specific low-level processes such as motor transcription through handwriting. 
The rapid interweaving of planning, translating, and reviewing at a high level 
of processing is reflected in multiple brain regions orchestrated almost 
concurrently. 

In the vast majority of the population, right-handedness implies that the 
left hemisphere controls many, yet not all, aspects of language 
comprehension and production. The dominant left hemisphere includes both 


Broca’s area in the frontal lobe and Wernicke’s area in a left posterior region. 


It was not a surprise, then, that producing language through handwriting 
extensively involved areas in the left parietal lobe, at least in right-handers, 
near but above Wernicke’s area. For example, it is well established using 
fMRI that portions of the left superior parietal cortex are recruited in the 
motor control of handwriting (Segal & Petrides, 2012). This region is active 
in composing an extended narrative via handwriting but not when it is 
produced orally (Brownsett & Wise, 2010). 

Planning ideas and translating them into coherent sentences — as 
opposed to the motor execution of handwriting — involve multiple regions of 
the language zone of the left hemisphere (Bechtereva et al., 2004). After the 
method of Bectereva et al., Howard-Jones, Blakemore, Samuel, Summers, 
and Claxton (2005) required adult writers to produce a creative story with a 
prompt of three words. Those were either related (e.g. magician, trick, rabbit) 
or unrelated (e.g. flea, sing, sword). They discovered that the prefrontal 
cortex of the right hemisphere is also recruited when writing a creative story 
using unrelated words. The reason for this is that the right hemisphere 
supports the divergent semantic processing needed to detect remote 
associations among the unrelated words (Beeman, 1998). 

The complex dance of planning, translating, and reviewing during pre- 
writing and drafting engages vast and diverse cortical areas of the brain (Shah 
et al., 2013). When asked to brainstorm how to complete a story from an 
opening few sentences provided by the experimenter, the strongest 
activations were centered in the left frontal brain regions, including Broca’s 
area for language generation plus an adjacent left prefrontal area for attention 
and cognitive control. Together, these activations reflected the demands on 
verbal working memory made by planning novel ideas for inclusion in the 


story. These same activations persisted when drafting the story compared 


with just copying a set of words. Actually drafting the story by hand also 
activated the left superior parietal lobe previously implicated in handwriting 
(Segal & Petrides, 2012) and the right frontal area observed by Howard-Jones 
et al. (2005) in creative semantic processing. In addition, Shah et al. (2013) 
found that creative writing activated the temporal lobe and the hippocampus 
on both sides of the brain; these are known to be involved in the retrieval of 
past experiences and words from long-term memory. 

The specific skills that professional writers bring to their job will be 
addressed next. This is not intended as an exhaustive list but rather as a 
selection of the key characteristics of professional writers and their writing 
skills. 


Writing Expertise 


Professional writers use language to activate extensive associations among 
verbal and imaginal representations in the reader (Sadoski & Paivio, 2001). It 
is the reader who finds a text meaningful, coherent, interesting, or persuasive. 
The words crafted by the author can potentially trigger the reader’s long-term 
memory representations from the bottom-up and motivate constructive 
processes from the top-down. That is to say, the text both relates to what a 
reader already knows and stimulates new thinking at the same time. Deep 
comprehension of a text calls upon the reader to construct a mental 
representation of what the text says and how this information integrates with 
other knowledge about the world (Kintsch, 1998). A shallow level of 
comprehension is fragmentary by comparison. Professional writers aim to 
engage the reader in such deep comprehension. How can their skill in doing 


so be characterized? 


Verbal Ability 


Construction of a locally cohesive text depends heavily on verbal ability. 
Knowledge of their language provides writers with the means to establish 
cohesive ties between one clause and the next by deploying multiple syntactic 


structures (McCutchen, 1984). Through careful word choice and grammatical 


markers the writer prompts the reader to establish coherence, making an 
inference if needed, in their mental representation of the text (Givon, 1995). 
Verbal ability also alters the rate as well as the quality of sentence 
production. As writers construct a sentence, they do so in bursts of words 
followed by pauses. Chenoweth and Hayes (2003) reported that more 
experienced writers (graduate students) generated twice as many words per 
burst (10-12) compared with less experienced writers (5-6 words for 
undergraduates). The graduate students’ superior verbal ability enabled the 
rapid retrieval of words, complete phrases, and complex grammatical 
structures from long-term memory, resulting in the large word bursts. 

Both vocabulary size and diversity in word choice correlate positively 


with judgments of writing effectiveness (Grobe, 1981). Of special importance 


is the use of concrete words that support both verbal and imaginal coding by 


readers during comprehension (Sadowski & Paivio, 2001), resulting in 


memorable text. Professional writers do not abolish abstractions from their 
texts; indeed at times abstract concepts can only be rendered using abstract 
words. They do, however, know how to provide readers with sufficient 
concrete references and graphic descriptions of context to make a text come 


alive in the reader’s imagination and memory. 


With growing expertise, writers use words fluently, precisely, and in 
sophisticated syntactic structures. Haswell (2000) examined essays written by 
college students as freshmen and compared them with those produced as 
juniors. He found systematic gains in vocabulary, the length of sentences and 
clauses, and the use of free modifiers that add grammatical complexity to 
their sentences. The length of the essays and their judged holistic quality also 
improved, reflecting enhanced fluency in composing and greater elaboration 
and substantiation of ideas in the texts. Of interest, all of these changes 
advance the college students closer to the levels of these variables observed 
in the writings of postgraduate professionals (Haswell, 1991). 

Similarly, Aull and Lancaster (2014) compared a large corpus of 
argumentative essays composed by freshmen with similar types of papers by 
juniors and seniors in college. The work of college students was also 
contrasted with scholarly articles published in peer-reviewed journals by 
professional academic writers. Freshmen boosted their commitment to their 
assertions, arguments, and conclusions by inserting words such as very, 
highly, strongly, totally, certainly, or undoubtedly far more often than did 
upper-level undergraduates and especially in comparison to the published 
texts of professionals. In the opposite direction, the use of hedges to qualify 
the scope or strength of a claim (e.g. apparently, essentially, generally, in 
many cases, primarily) was 85 percent more likely in the work of 
professionals compared with freshmen; again, the advanced undergraduate 
writers fell between these two extremes. As with Haswell’s (2000) findings, 
the gains in writing skills shown by the juniors and seniors moved them 
closer to the frequencies of use for boosting and hedging seen in professional 


writers. 


Managing the Cognitive Load 


Composing can place severe demands on working memory because the task 
requires temporarily maintaining numerous mental representations in 
planning ideas, translating sentences, and reviewing the results. The verbal 
protocols collected by Flower and Hayes (1980) indicated that: “A writer 
caught in the act looks ... like a very busy switchboard operator trying to 
juggle a number of demands on her attention and constraints on what she can 
do” (p. 33). These demands have been further documented by measuring the 
time needed to respond to a secondary auditory probe during writing. It takes 
longer to detect the probe and respond while one is composing, compared 
with listening for the probe as a sole task. The degree to which responding 
slows down while writing reflects the concurrent demands of composing 
processes on working memory. Of interest, writing text causes very large 


delays in responding to the probes for college-level writers, much larger than 


that observed with learning and reading tasks (Kellogg, 1994). The delays 
when writing were comparable to those observed in earlier research with 
chess experts evaluating multiple moves in the middle stages of a game 
(Britton & Tessor, 1982). 

The ability to fluently combine the high-level cognitive processes of 
composition with the low-level processes of motor output emerges only in 
advanced writers. Almargot, Plane, Lambert, and Chesney (2010) measured 
both the movements of the pen during handwritten composition and the 
movements of the eyes to track composition processes in case studies of 
middle and secondary school writers, a graduate student, and a professional 


writer. Only the professional and, to a lesser extent the graduate student, 


could adeptly perform low-level transcription and mechanics as well as high- 
level planning, sentence generation, and reviewing in a relatively automatic 
way. 

Neuroimaging data from experienced authors with professional training 
in creative writing also reflect changes in how experts are composing 
(Erhard, Kessler, Neuomann, Ortheil, & Lotze, 2014). The brain activations 
were less widespread and largely lateralized to the frontoparietal-temporal 
network of the left hemisphere during creative composition. For example, 
Broca’s language area and the nearby area for attention and cognitive control 
revealed greater activation in college students accepted into and trained in a 
highly selective university program for creative writers compared with 
control students without such preparation. In testing the same two groups, 
Lotze and colleagues (Lotze, Erhard, Neumann, Eickhoff, & Langner, 2014) 
further observed that the trained creative writers showed less connectivity 
between the left and right hemispheres during the resting state in an area of 
the frontal lobe associated with language processing in comparison with the 
control participants. Instead, the advanced writers showed an increase in 
resting state functional connectivity within the right hemisphere. Intriguingly, 
then, it may be that experienced professional writers compose with less 
cooperation between the left and the right cerebral hemispheres. 

A variety of cognitive stratagems help the writer to cope with the 
cognitive demands of text production. Pre-writing, drafting, and revising are 
essential strategies. Preparing an outline as a pre-writing strategy enables the 
writer to focus attention on translating ideas into text while drafting and can 


enhance productivity (Kellogg, 1988). Individuals who write in organizations 


as part of their job responsibilities frequently report the need to plan a lot 


mentally (98 percent), create notes and lists (95 percent), and prepare outlines 


(73 percent), especially for serious writing tasks that do not fit a routine 
schema (Couture & Rymer, 1993). 
Highly successful writers have reported a wide range of pre-writing 


Strategies that can be expressed as styles (Cowley, 1958; Plimpton, 1963, 


1989). “Beethovians” engage in few pre-writing activities and prefer to 
compose rough first drafts immediately to discover what they have to say 
(Bridwell-Bowles, Johnson, & Brehe, 1987). Their drafting necessarily 
involves many rounds of revision. By contrast, “Mozartians” delay drafting 
for lengthy periods of time in order to allow time for extensive reflection and 
planning. They may also plan mental pre-text that is later recalled and written 
down as a first polished draft. A variety of notational methods are used to 
extemnalize plans during pre-writing, including tree diagrams, flow charts, 
boxes, arrows, doodles, and scribbles as well as lists and outlines (John- 
Steiner, 1985). 

Preparing a rough draft can help writers to reduce the number of 
processes juggled simultaneously by dissociating the author from an editing 
mindset (Glynn, Britton, Muth, & Dogan, 1982). An fMRI study of poetry 
composition captured this loss of cognitive control. Published poets with at 
least a year of training in a master of fine arts graduate program showed more 
deactivation of the cognitive control regions in the dorsolateral prefrontal 
cortex mediating executive attention and working memory compared with 
less experienced poets while first generating the text (Liu et al., 2015). Only 
later during revising a subsequent draft did these control areas come back on 
line. “Letting go” during a first draft may be especially critical in poetry 
composition and other creative writing. 

In the computer-based environments of today’s workplace, composing a 


first draft can be speeded by reusing text from the author’s previous works 


stored digitally. Professional writers can rapidly assemble a first draft, 
copying, pasting, and revising the recycled text to suit the goals of the new 
text. Leijten, van Waes, Schriver, and Hayes (2014, p. 297) provided a case 
study of a proposal writer and observed that in a prolonged first session: “He 
copied and pasted information from related project proposals and rewrote 
those bits and pieces seemingly instantly” (p. 297). Of the four additional 
sessions needed to complete the proposal, only the second involved reading 
and revising the text, with the others devoted to budget preparation, 
assembling a team, and thinking about how to present the material to his 


boss. 


Domain Expertise 


As noted earlier, writing expertise is highly domain-dependent. In an 
important sense, then, the professional writer may be a rare individual 
indeed, when conceived as a generalist capable of writing across any number 


of domains (Carter, 1996). Outstanding journalists might be lost if they had 


to compose a scientific report. A scientist may be equally adrift in trying to 
write for the general public. Even within a profession, a writer often 
specializes in a specific rhetorical context. For example, a journalist who 
specializes in movie reviews for the New York Times might well perform 
more like a novice in writing market analyses for the Wall Street Journal. 
There are indeed diverse, highly specific skills required by journalism, 
exposition, creative fiction, business writing, or technical writing. 
Professionalism in any one of these kinds of writing requires a progression 
beyond general writing skills, such as outlining, fluent sentence generation, 
and effective topic sentences, to domain-specific knowledge and skills. 
Professional writing courses in colleges and universities are increasingly 
attuned to the fact that “what students take with them across the 
academic—workplace boundary is less a set a explicitly transferable skills and 
more a generalized rhetorical capacity that enables them to successfully adapt 


to new rhetorical situations” (Read & Michaud, 2015, p. 428). Learning how 


to learn to write in a specific domain outweighs other goals of the instruction. 

The expert must master the unique rhetorical style required in a given 
domain. Psychologists, for example, learn the rhetorical style mandated by 
the American Psychological Association Publication Manual, which 


contrasts with the Chicago Manual of Style used in the humanities (Madigan, 


Johnson, & Linton, 1995). Psychologists’ conclusions must be hedged so as 
to avoid stepping beyond the limits of the data, creating an air of uncertainty 
that is atypical in the humanities. Disagreements are couched in terms of 
differences about appropriate methods or interpretations of data, never in the 
personal terms at times seen in literary criticism. Similarly, Dahl (2009) 
examined the research articles published in economics journals to discover 
the rhetorical style employed at least in this discipline, if not broadly, in 
academic writing. Specific expressions were used to lead the reader to the 
conclusion advanced by the author(s). For example, deploying the present 
tense to express an argument in both the introduction and the conclusion of 
the article (e.g. We find ... /We argue ...) is a way to convince the reader that 
the findings are applicable both now and in the future and are thus of general 


importance. 


Rapid Access to Long-Term Memory 


Domain-specific knowledge allows experts to escape, in part, the constraints 
on working memory that hinder effective writing in novices (McCutchen, 
2000). Expertise allows the rapid, facile retrieval of representations from 
long-term memory as they are needed rather than requiring the thinker to 
maintain everything actively in short-term working memory (Ericsson & 


Kintsch, 1995). The ability of domain experts to use long-term memory as a 


kind of working memory could be the critical advantage they have over less 
knowledgeable writers in achieving fluent production (McCutchen, 2000). 
For example, college students who scored highly on a test of baseball 
knowledge responded to auditory probes as they wrote narratives of a half- 
inning significantly faster than was observed for control participants with 
little knowledge of the game (Kellogg, 2001). 

Rapid retrieval from long-term working memory allows experts to take 
into account more ideas in planning a text relative to novices. For example, 
Geisler, Rogers, and Haller (1998) asked practicing software engineering, 
advanced software engineering students, advanced technical communication 
students, and advanced chemistry students to list all the issues that needed to 
be resolved in designing an automated bus ticket issuing system. The 
engineers listed 62 percent more ideas than the average of the other three 
groups combined. Compared to the advanced chemistry students, who 
performed the worst, the practicing engineers retrieved markedly more 
information about system issues (71 percent) and business issues (83 


percent), for example. 


Managing the Emotional Challenges 


Professional writers, as with other practitioners of the creative arts, must be 
self-motivated to commit long hours to a lonely task of working with ideas 
and language rather than with other people. As Hayes (1996, 2012) 
postulated, affective and motivational factors loom large in successful 
writing. Indeed, the emotional demands of writing can constrain a writer as 
severely as, if not more so than, the cognitive demands (Brand, 1989). The 
need to produce extended texts of thousands and tens of thousands of words 
can deplete the motivation of would-be professional writers. Experienced, 
successful writers thus must learn to self-regulate their emotions and 
behavior to stay on task and complete the work. An unwillingness to write 
can easily prolong the composition process and result in failures to meet 
deadlines. 

Major breakdowns in professionals’ efforts to regulate their emotions 


and behavior can result in writer’s block (Boice, 1994), which is defined as a 


persistent inability to put thoughts on paper. Contrast the productivity and 
success of a scientist with 1,200 publications to his credit with another “who 
seemed never to get to writing up the work he did” and disappeared from the 


field (Sternberg, 2012, p. 449). Whether one publishes or perishes can turn on 


emotional and motivational factors involved in text production rather than 
purely cognitive abilities. Consider that the main hindrances to productive 
writing in PhD candidates are: (1) emotional blocks and procrastination, (2) 
an attitude of perfectionism, and (3) a belief that writing talent is innate rather 


than developed through experience and practice (Lonka et al., 2014). 


How do writers go about managing the ups and downs of their craft and 
stay motivated to finish an often lengthy project? Four approaches emerge: 
(1) “losing” themselves in their works, (2) engineering their work 
environment, (3) adhering to a work schedule, and (4) practicing motivational 


rituals. 


Flow States 


Capable writers from the level of high school essay writers (Larsen, 1988) to 
published novelists and poets (Perry, 1996, 2005, 2009) at times get lost in 
their work. Hours of work can pass with minimal awareness of the passage of 
time. Csikszentmihalyi (1990) referred to such absorption and the positive 
affect associated with it as a flow state of consciousness. Flow can be 
produced when an individual’s aptitude and skills are well matched by the 
demands of any task and writing is no exception. Neither bored by a trivial 
task nor frustrated by a task beyond one’s current cognitive and emotional 
capacity, a writer in flow focuses intensely on the task and loses self- 
awareness. Although unproven, many writers believe that their best work 
emerges from flow states of absorption in the task (Perry, 2009). 

Schere (1998) contrasted university professors of creative writing, fine 
arts, and engineering on a drawing versus writing task. As predicted, the 
creative writers reported a higher level of positive mood after completing the 
writing task compared with the drawing task. Brand and Leckie’s study of 


professional writers (1989) also concluded that the opportunity to engage a 


task that fits the writer’s capabilities enhances positive mood. 
Perry (1996, 2005, 2009) interviewed 36 published poets and 40 


published short-story writers and novelists about the conditions that lead to 


flow. Entry into flow, she concluded, is chiefly facilitated by having a strong 
motivation to write. Believing that the writing task is intrinsically important 
is sufficient for many professional writers. Some evidence suggests that a 
writer’s creativity can actually be diminished by extrinsic rewards (Amabile, 
1985), but Perry found that pay and other extrinsic motivators can also 
stimulate flow for a least some professional writers. Another unique feature 
of writing in flow concerns the role of feedback. Normally a flow state 
requires getting feedback, enabling one to feel a sense of mastery over the 
demands of the task (Csikszentimihalyi, 1990). However, this feedback is 
either internally generated or unnecessary in writing, given that external 
feedback is usually delayed until at least after a draft is completed (Perry, 
2009). The frequency of flow states varies across individuals (Perry, 2009): 
for some writers “flow happens whenever they write, whereas for others, it is 
more elusive” (Perry, 2009, p. 38). 

Although the positive affect of flow states can characterize the work of 
professional writers, they also face frustrations, anxieties, and other negative 
emotions when the demands of the task temporarily exceed their capacities. 
As Perry (2009) observed, a flow state can be “more often wished for than 
dependably achieved” (p. 24). Writer’s block can be one consequence 
whereby production is sharply curtailed or stopped altogether (Boice, 1994). 
Perry (1996) documented that some writers force themselves during these 
periods “to produce line after line ... and the product of such reluctant 
writing sessions is often not up to what the writers consider their standard” 
(p. 275). Thus, judging from Perry’s interviews, creative writers manage to 
tolerate negative emotional states as well as enjoying the positive experience 
of flow. It is also important to recognize that procrastination can be beneficial 


when it reflects a real need to take more time (Murray, 1978). Extensive pre- 


writing activities can be a way for professional writers to digest and 


understand their subject thoroughly before beginning a first draft. 


Environments, Schedules, and Rituals 


Where, when, and how writers work show enormous variability. However, 
many writers develop habitual ways of approaching their work such that they 
become necessary conditions for effective knowledge use. Perry’s (1996, 
2005, 2009) interviews with established creative writers and surveys of 
university faculty who produce scholarly publications (Boice & Johnson, 


1984; Hartley & Branthwaite, 1989; Kellogg, 1986) converge on three points. 


All of the idiosyncratic habits of professional writers (1) focus attention 
inward by eliminating distractions, (2) may alter consciousness to facilitate 
entry in a flow state, and (3) help regulate the writer’s emotional state to keep 
at the task. 

For example, whereas some writers report that they find a quiet 
environment useful, others preferred background music or the bustle of a 
café. Whereas some must write with a word processor, others prefer longhand 
or a typewriter. Many choose to write at the same time each working day, but 
individuals differ from morning, afternoon, evening, late night, to early 
morning preferences. Running or walking help some writers think through 


problems while away from the writing table (Kellogg, 1986; Oates, 2003). 


Others use meditation, coffee, cigarettes, alcohol, or other drugs to alter 


consciousness in the service of writing (Piirto, 2002). Some writers find 


daydreaming and even dreaming at night conducive to their work (Epel, 
1593). 


Work sessions of one to two hours correlate with productivity in 
scientific writing, but the relationship is weak (r = 0.22) and the variability is 
large, with some individuals writing four or more hours at a time (Kellogg, 
1986). Successful poets also typically write for one or two hours, whereas 
most novelists typically report longer sessions of two to three or even four to 
six hours (Perry, 1996, 2005). 


Blocking 


Writer’s block can impede, if not end, the career of a professional writer. 
Boice (1985) found the thoughts of blocked academics were characterized by 
procrastination, dysphoria (e.g. burnout, panic, obsessive worries), 
impatience (e.g. thoughts of achieving more in less time or imposing 
unrealistic deadlines), perfectionism (e.g. thoughts reflecting an internal critic 
who allows no errors), and evaluation anxiety (e.g. fears of being rejected). 
Of interest, unblocked as well as blocked writers fretted about how difficult 
and demanding the task was for them. 

Thus, professional writers know not only how to enter flow, but how to 
endure the intense negative feelings that often accompany the early stages of 
writing a text, when all can seem hopelessly incoherent. Learning to manage 
the emotional ups and downs of writing is important for writers to avoid 
burning out and perhaps ending their career prematurely. Self-regulation 
through daily writing, brief work sessions, realistic deadlines, and 
maintaining low emotional arousal help professional writers stay with the 


task for the weeks, months, and years required (Boice, 1994; Zimmerman & 


Risemberg, 1997). Binge writing — hyponomanic, euphoric, marathon 


sessions to meet unrealistic deadlines — is generally counterproductive and 
potentially a source of depression and blocking (Boice, 1997). 

To summarize, there is a wide variety of skills and qualities that 
professional writers bring to their daily work. In the final section, the steps 
involved in becoming an expert writer are considered. The stages of writing 
development leading to a high degree of expertise are outlined along with the 
role of deliberate practice. Finally, the rule that expertise requires ten years of 


deliberate practice is considered with respect to written composition. 


Skill Acquisition 


Literacy is a fundamental goal of schooling in contemporary societies 
worldwide. Thus, unlike medical diagnosis and other professions for which 
practice begins in early adulthood for a select few, writing development starts 
in early childhood for many. The foundations of literacy are established in 
toddlers with letter and word recognition and scribbling. By the time a child 
enters school, practice in handwriting is well underway, and children as 
young as 4 years of age distinguish writing from drawing (Lee & Karmiloff- 
Smith, 1996). 


Stages of Development 


By adolescence, children have devoted about ten years to mastering the 
mechanics of handwriting and spelling, given the slow pace of cognitive 
maturation and the sporadic nature of writing practice. Approximately during 
this same time frame, they advance from thinking in term of concrete events 
in the here and now to thinking in hypothetical, abstract terms. From studies 
that tracked the cohesion of texts produced by children at different ages, 
Scinto (1986) concluded that this advancement from concrete to abstract 
thinking is essential for writing cohesive texts. Although speech is proficient 
by the age of 6, written fluency does not catch up until around the age of 12 
years (Bereiter & Scardamalia, 1987). 

Even after a child is able to fluently produce texts, the manner in which 
processes and representations are managed in working memory requires 
significant, lengthy development. The written production strategy of children 
is a simple one of egocentric knowledge telling (Bereiter & Scardamalia, 
1987). An idea is retrieved from long-term memory and then written down. 
This process continues until the writer has no more ideas to retrieve and 
communicate. The heavy demands made on working memory by handwriting 
alone make it difficult for young writers to do much more than retrieve and 
translate (Bourdin & Fayol, 1994; McCutchen, 1996). The author can 


maintain a representation of what he or she intends to write, but no more than 


this. 
By contrast, Bereiter and Scardamalia (1987) characterized adult writing 
as knowledge transforming. What the writer thinks is transformed by the act 


of composition. The adult writer reads the evolving text and develops a text 


representation of what it says to the author. Reflection on the existing text 
can prompt the writer to restructure the ideas stored in long-term memory, 
elaborating and reorganizing what the writer knows about the topic. 
Knowledge telling often produces a string of assertions linked by the 
conjunction “and.” In contrast, knowledge transforming yields complex 
argument structures and the use of many cohesive devices to link clauses in a 
paragraph. The adult writer, therefore, maintains representations of the 
author’s current intent and what the text says from the author’s perspective. 
After mastering handwriting and achieving written fluency, approximately a 
decade of practice is needed to progress from knowledge telling to 
knowledge transforming. Bereiter and Scardamalia (1987) turned to graduate 
student writing to illustrate knowledge transforming. 

Many additional years are needed to acquire the domain-specific 


rhetorical skills, through practice at crafting knowledge for a specific 


audience (Kellogg, 2008; Rymer, 1988). Knowledge crafting entails shaping 
a text so that the reader finds it comprehensible and convincing; it demands 
that the author see her words from the third-person perspective of the reader. 
Besides the author representation and the text representation, a reader 
representation must also be held in working memory (Traxler & Gernsbacher, 
1993). In knowledge crafting, the writer must be able to take the perspective 
of an imagined reader and compare this representation to both the author’s 
current ideas and the author’s comprehension of the text. In doing so, 
professional writers aim to craft their knowledge, through their writing, to the 
needs of a specific audience. Knowledge crafting requires not only adeptly 
juggling planning, sentence generation, and reviewing but also maintaining in 
working memory three different representations. Professional revision of the 


text can thus go beyond simply detecting mismatches between the author’s 


intent and the text as it reads to the author. Rather, the professional writer 
aims to see the text as a reader would see it. 

Revision is often minimal in young writers and even in college students, 
with corrections focused primarily on word substitutions and surface-level 
changes (McCutchen, 1996; Sommers, 1980). In professionals, by contrast, 
Sommers (1980) found that journalists, editors, and academics review their 
texts for the “form or shape of their argument” while holding in mind “a 
concern for their readership” (p. 384) and “imagine a reader (reading their 
product) whose existence and whose expectations influence their revision 
process” (p. 385). Academic writers in particular must craft their articles so 
as to anticipate readers’ reactions to their arguments and head off objections 
so as to be persuasive. Hyland (2001) examined 240 published articles from 
leading journals in ten disciplines and found markers of the dialogue that 
emerges in the text between the writer and reader. For example, the writers 
posed real and rhetorical questions to engage the reader. They made direct 
references to the reader and to shared knowledge, and used second-person 
pronouns and asides addressed to the reader. 

Even so, as every professional writer knows from the criticism by their 
editors and reviewers during the publication process, it is enormously 
challenging to foresee every possible misreading of their text. As Ong (1975) 
correctly observed, the ability to take the reader’s perspective fully into 
account is what separates the beginning scholar from the mature scholar. 

Failures to craft knowledge for readers abound, from poorly written 


technical documentation (Hayes & Flower, 1986) to jargon-filled legal 


documents that are incomprehensible to the general public (Hartley, 2000). 
Even professional writers hired to improve the clarity of such documents do 


not always succeed (Duffy, Curran, & Sass, 1983). Still, the professional is 


far better prepared to cope with the challenge of knowledge crafting than the 
less experienced writer. As Schriver (2012) pointed out, a professional writer 
can draw upon an immense number of language patterns stored in long-term 
memory — far in excess of the 100,000 patterns needed in expert chess play — 


in order to revise a text to communicate with a specific audience. 


Deliberate Practice 


Ericsson, Krampe, and Tesch-R6mer (1993) defined the characteristics of 
deliberate practice as: (1) effortful exertion to improve performance, (2) 
intrinsic motivation to engage in the task, (3) practice tasks that are selected 
by a teacher to be within reach of the individual’s current level of ability 
within a limited amount of practice, (4) feedback that provides knowledge of 
results, and (5) gradual refinement from many repetitions. They contrasted 
deliberate practice with work activities for pay or other external rewards and 
enjoyable play activities that have no goal for improving performance. 
Deliberate practice can only be sustained for a limited amount of time each 
day because of the effort involved. 

Evidence of practice in writing meeting several of the criteria for 
deliberate practice emerged from several ethnographic survey studies of 
professional writers (Henry, 2002; Kellogg, 1986; MacKinnon, 1993; 
Paradis, Dobrin, & Miller, 1985). Despite a diversity of workplace cultures, 
professionals improve their writing skills through writing often, receiving 
feedback from colleagues, and devoting high levels of cognitive effort to 
their work. High levels of practice can also be seen in the daily work 


schedules of prolific creative writers (Cowley, 1958; Plimpton, 1963). 


Because the work requires such high levels of effort, successful writers 
typically schedule only a few hours per day for composing, and avoid binges 
that lead to exhaustion (Boice, 1994, 1997). 

Garrett and Moltzen (2011) stressed the importance of intrinsic 
motivation to practice both in eminent adult writers and in young children 


with a gift for writing. They concluded from their study of 32 young gifted 


writers that “their primary motivation to write came from within” (p. 177). 
Although in the early school years, the positive feedback and encouragement 
of parents and teachers was helpful, school-based writing assignments during 
the adolescent years distracted many of the students from practicing through 
their personal writing 

In developing professional expertise, it has long been known that writers 
need coaching through feedback. Apprenticeships in creative writing 
programs and schools of journalism rely on this method of practice. The Iowa 
Writer’s Workshop, the earliest creative writing degree program in the United 
States, has from the outset emphasized learning by doing and immersing 
writers in feedback from others (Adams, 1993). As early as 1890, writers’ 
clubs formed in Iowa City where students and faculty could read and critique 
the work of each other and practice their craft. Journalists, too, have always 
learned by doing, first in newspaper offices and later in formal university 
programs. From its beginning in 1908, the University of Missouri’s School of 
Journalism has run a daily city newspaper for students to practice their skills 
with instruction and feedback from the faculty. 

Well-tailored, intensive practice within a single discipline successfully 
advances the writing skills of college students. Johnstone, Ashbaugh, and 
Warfield (2002) compared accounting students who took two intensive 
business writing courses in their junior year followed by two more such 
courses in their senior year with business students who lacked such extensive 
training and practice. A standardized test of writing skill given in the 
sophomore year showed the two groups to be equal. The assignments in the 
writing intensive courses required the students to compose as accounting 
professionals to a professional audience. They received thorough feedback 


regarding grammar, organization, professionalism, technical accuracy, and 


analysis quality. A final standardized test in the senior year showed steady 
and reliable gains for the intensive writing students compared to the control 
group that showed no gains at all. 

Well-known fiction writers have reported their valued practice 
techniques. As a college student, Joyce Carol Oates would write a novel in 
longhand, and then turn the pages over, writing another novel on the flipside. 
Both novels would then be tossed in the trash. Since high school she began 
“consciously training myself by writing novel after novel and always 


throwing them out when I completed them” (Plimpton, 1989, p. 378). Her 


practice books would be modeled after specific works by authors she 
admired, such as Hemingway’s In Our Time. Norman Mailer (2003) credited 


his eventual success as a writer to self-motivated practice: 


I think from the time I was seventeen, I had no larger desire in life than 
to be a writer, and I wrote a great deal ... I learned to write by writing. 
As I once calculated, I must have written more than half a million words 
before I came to The Naked and the Dead ... 


(pp. 13-14) 


Purposeful practice is an important aspect of developing writing skills in 
primary, secondary, and college education (Kellogg & Whiteford, 2009). 
Even for a college graduate with strong writing skills, the necessity of further 
practice remains. The budding professional writer needs to continue training 
through an apprenticeship with an experienced mentor. To illustrate, graduate 
students pursuing the sciences benefit from a cognitive apprenticeship in 
writing courses built on modeling, coaching, collaboration, and providing 


scaffolds that lessen the load on working memory while learning (Ding, 


2008). They train to write grant applications to the National Institutes of 
Health and learn from models of high quality writing through journal clubs 


where students orally present published research from the top tier journals, 


such as Science or Nature. 


The Ten-Year Rule 


Studies of chess players (Simon & Chase, 1973), musical composers (Hayes, 


1985), and other domains (Ericsson et al., 1993) have suggested a rule of 


thumb that it takes at least a decade of intensive preparation to achieve 
excellence. Because of early instruction in literacy skills in primary school 
education, ten years is indeed an underestimate. But excluding the early 
preparation, the ten-year rule seems to hold for professional writers. 
Wishbow (1988) examined the biographies of 66 poets listed in the Norton 
Anthology of Poetry, locating their approximate starting date for reading and 
writing poetry. The earliest work to appear in Norton’s came during the ten 
years after this date or later for 83 percent of the sample. Similarly, T. S. 
Eliot wrote his masterpiece, The Waste Land, in his early thirties, about a 


decade after composing his first published poem (Gardner, 1993). Kaufman 


and Kaufman (2007) also reported that among 215 contemporary fiction 
writers the progression from first publication to best publication required an 
average of 10.6 years albeit with wide variability. As the authors noted, 
“Norman Mailer took 11 years to produce his best work, John Irving took 16 
years, and Don DeLillo took 26 years” (p. 120). 

Not surprisingly, the earlier the writer starts the better. Childhood story 
writing was so commonly mentioned in Henry’s (2002, p. 37) ethnographies 
that “people who were attracted to writing after childhood may even refer to 
themselves as ‘late bloomers’.” A study of 986 creative writers found a 
significant correlation between the age of first publication and the number of 


works published in total for poets (Kaufman & Gentile, 2002). Replicating 


and extending Wishbow’s (1988) findings, both poets and fiction writers 


developed mechanics and cognitive writing skills for 15—20 years before first 
publishing. Only 10 percent of the sample published prior to the age of 21. 
About half (49 percent) first published in their twenties and the remainder in 


their thirties or later. 


Conclusion 


Professional writing reflects not just general writing ability but also expertise 
in a particular genre and domain. Thus, there are really many specific kinds 
of professional writers, but distinctive features apply to them all, such the 
skilled use of language and in-depth knowledge of a specific domain of 
writing. The capabilities to cope with the heavy cognitive as well as 
emotional demands of written composition further distinguish the 
professional writer. The ten-year rule of expert skill acquisition applies to 
writers and, if anything, underestimates the number of years of practice 
required for professional levels of achievement. Much has been discovered 
about the skills of professional writers that can benefit those aspiring to the 


role. 
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Introduction 


There are approximately 3.5 million full-time elementary and secondary 
teachers working in US public and private school classrooms (US 
Department of Education, 2016). Given their potential to impact the lives of 
young people, it would seem to be of great importance to understand what 
makes one teacher more effective than another, what expertise in teaching 
looks like, and how it develops. Despite the fact that teaching is one of the 
oldest and largest of human professions, however, we still lack a clear 
conception of what it means to be an expert teacher. This is not because 
researchers have not tried to pin this down. They have, and we will try in this 
chapter to make a contribution to this effort. But it is worth discussing, at the 
outset, why these questions have proven so difficult to answer. 

Perhaps the biggest challenge to understanding expertise in teaching is 
that teaching is not an individual endeavor in which the teacher him- or 
herself is the only actor. Teaching is a complex system of interacting 
elements, and effective teaching requires that all of these elements work 
together to produce the desired outcomes. Later we will consider the nature 
of this system. But for now, consider simply that teachers cannot achieve 
their goals without the cooperation of students. Chess experts don’t require 
the cooperation of the chessboard. The chessboard, the musical instrument, 
and so on are invariants in many domains of expertise. But in education, 
teachers are in a very real sense at the mercy of students, and policy-makers, 
and curriculum designers, and so on. This is something we will need to 


address as we consider what it means to be an expert teacher. 


Another obstacle to understanding expertise in teaching is the “pseudo- 
expertise” one develops as a student. Before anyone starts their formal 
training as a teacher, they already have experienced well over 10,000 hours 
as students in classrooms, making teaching the profession with the most 
intensive and lengthy apprenticeship of any. One consequence of this 
experience is that everyone in our society, including teachers, thinks they 
already know what an expert teacher is, without any serious consideration of 
the research. This leads to bias in research on teaching, with a distinct lack of 
research designed to investigate theories that “everyone knows aren’t true.” It 
also may reduce variation in teaching methods within our culture, which 
makes it more difficult to explore alternatives. 

Finally, our work is hampered by a lack of a consensus on the aims of 
education. We have a wide array of desired educational outcomes, yet 
teaching practices that prove effective for one outcome might be ineffective, 
or worse, for others. Similarly, practices that yield impressive short-term 
results (e.g. high scores on year-end standardized tests) may have negative 
effects on long-term outcomes such as career advancement and satisfaction 
decades later. Work by Jackson and colleagues (Jackson, 2012; Jackson, 


Rockoff, and Staiger, 2014), for example, has shown that teachers who 


produce the strongest gains on achievement tests are not the ones who 
succeed at reducing absences and suspensions, variables shown to predict 
future educational attainment and adult earnings. This complicates the 
straightforward strategy of studying teaching expertise by studying 
acknowledged experts. 

Although it is tempting to rally round a single measure of student 
outcomes (e.g. the currently popular “value added” models of student 


achievement), we must be careful not to get too focused on the metric instead 


of on the underlying process the metric is intended to reflect. Campbell 


(1979) warned of this in what came to be known as “Campbell’s law”: 


The more any quantitative social indicator is used for social decision 
making, the more subject it will be to corruption pressures and the more 
apt it will be to distort and corrupt the social processes it is intended to 


monitor. 


(Campbell, 1979, p. 85) 


These concerns don’t invalidate the effort to understand and measure student 
educational outcomes or to understand how teachers develop the skill to 
promote these outcomes, but they do help explain why understanding 
teaching expertise is not a simple matter of identifying and studying 
acknowledged experts. 

In this chapter we try to take a broader approach to understanding the 
nature and development of expertise and expert performance in teaching. 
Because the literature on expertise in teaching is limited, we instead try to 
integrate a number of ideas and findings from literatures as diverse as cross- 
cultural comparisons of teaching, cognitive psychology, and systems 
improvement, among others. At a minimum, we hope to start a new 
conversation about what expertise looks like in the highly complex domain of 


classroom teaching. 


The Nature of Teaching 


A Definition of Teaching 


Coach John Wooden famously said, “Everyone’s a teacher, to someone” 


(Gallimore & Tharp, 2004, p. 119). The ubiquity of teaching means that we 


need to restrict the domain we seek to describe. In this chapter we will 
narrow the focus somewhat to include only classroom teaching. Despite this 
narrow focus, classroom teachers make up the largest segment of public 
sector employment, with 3.5 million teachers in the United States (US 
Department of Education, 2016) and more than 29 million in the world 
(UNESCO Institute for Statistics, 2015). 

We base our definition of teaching on one offered by Lampert (2003; 
see also Ball & Forzani, 2007). Lampert sees teaching as “working in 
relationships,” specifically, the relationships among the three core elements 
of a classroom lesson: the teacher, the students, and the content that is being 
taught. Teachers must manage the content and the students, but most 
importantly they must manage the relationship between students and content, 
over time. These relationships define the “problem spaces” in which teachers 
work. So, for example, teachers must relate to students, and must collaborate 
with students to get work accomplished, which generally means getting 
students engaged with studying the content. Problems arise in the 
management of each of these relationships, and teachers, over time, develop 
routines for handling the problems that recur. 

Although we find this model to be a useful starting place for a full 
definition of teaching, we will broaden it in three ways. First, teaching always 
implies some goal for students (e.g. a learning goal) and some sequence of 
events designed to achieve that goal. Although some educators may prefer a 


more democratic view of relationships within the classroom, we believe it is 


important to see the teacher as an actor with a specific agenda for what she 
wants students to learn. This goal-directed nature of teaching was cited 
explicitly in some early definitions of teaching. For example, Thorndike 
(1906) defined teaching as the methods used to help students achieve the 
learning goals valued by society. The importance of making this agenda 
explicit has grown over the past 10 or 20 years as states and districts have 
focused more on creating and implementing clear academic content and 
performance standards for what students should know and be able to do at 
each grade level. Despite the recent influence of national and local standards, 
however, the teacher must still decide what learning goals the students in her 
class can and should achieve — at the end of the lesson, at the end of the unit, 
or at the end of the whole school year. 

A second modification we make to Lampert’s model of teaching is to 
broaden “teaching” to include the planning and reflection that go on before 
and after the lesson. It is common in the United States to overemphasize the 
importance of what happens during the lesson — the classroom performance 
of the teacher — and de-emphasize the intellectual work outside the classroom 
— what happens during planning and reflection. In fact, the effectiveness of a 
classroom lesson can be determined as much or more by the plan as by the 
on-the-fly decisions made by the teacher during the lesson. Planning is a 
teacher’s single most powerful leverage point for improving the quality of 
what happens during a classroom lesson. Likewise, the teacher’s later 
reflection on, and analysis of, problems that occur as the lesson unfolds can 
lead to revisions in the plan for next time, revisions that yield improvements 
in instruction. Reflection, or analysis, as we will call it, can be thought of as 
planning for the future informed by evidence gathered during the lesson. 


A final element in our definition of teaching is time. Classroom teaching 


is highly constrained by time. Teaching is often implemented as a series of 
lessons, and these lessons are separated in time. Each lesson is often limited 
to a set amount of time. And there are only a finite number of lessons in the 
school year to cover the prescribed curriculum. Thus, in some sense, planning 
of individual lessons and sequences of lessons is a zero-sum game: as new 
activities are added, others must be deleted. 

The centrality of time in teaching becomes even clearer when we 
consider students. Berliner (1990) noted that the amount of time students 
spend actively engaged in learning is often a small fraction of the time they 
spend in school. The amount of such engaged time can vary greatly across 
classrooms, and this can be a strong predictor of student learning (Fisher et 
al., 1980). Teachers’ skillful management of time is a key factor in 
determining student learning. That management, and the planning and 
decisions that undergird it, are an important aspect of teaching. 

Thus, we offer this definition of teaching: Teaching consists of the 
interactions of teachers, students, and content, in classrooms, that are 
intended to achieve some goal or goals for students, within a specified period 
of time (e.g. a classroom lesson), together with the planning that takes place 


before, and the analysis that takes place after, the lesson. 


Teaching Is a System 


More than many domains of expertise, teaching is a complex system with 
many moving parts. An expert pianist, for example, can have near total 
control over the product of her expertise, reliably performing beautiful music 
time after time. She can depend on a great deal of constancy in the 
environment. Unlike students, the keys on the piano on which she performs 
can be counted on to stay in place and respond consistently; the Mozart 
concerto will be the same each time she performs it; and the audience will 
apply a common set of standards as they appreciate and evaluate her work. 

Teaching, in contrast, is constrained by a number of variables, many 
beyond the teacher’s control. First there is the matter of students: a teacher 
can succeed only if students cooperate and engage in the tasks and activities 
the teacher assigns. According to Cohen, a leading education researcher, this 
is one of the key predicaments that define the teaching profession (Cohen, 
2011). But it is not just the students that co-determine the results of teachers’ 
work: curriculum, textbooks, assessments, policies, parents, peers, and so on, 
all interact and contribute to the nature and outcomes of the system we call 
teaching. 

The fact that teaching is a complex activity that must be considered from 
a systems theory perspective presents challenges to those who would describe 
and study expertise in teaching. A teacher may appear to be an expert in one 
environment, yet seem more like a journeyman when observed in a different 
environment. The success of a lesson depends in part on a teacher’s 
preparation and planning and her ability to assess students’ understanding 


and respond to their needs. But it also depends on factors that are largely out 


of her hands, such as the previous experiences of those students as well as 
institutional culture, routines, and support for learning. 

The question of whether expert performance “belongs” to the expert or 
to the organizational context is a matter of continuing controversy. Mueller 
and Dyerson (1999) have argued that expertise in complex institutions 
requires organizations that can develop and take advantage of individual 
skills; absent that, human experts may fail to make a_ contribution 
commensurate with their abilities. Others, such as Hoffman (1998), have 
argued that the question of whether knowledge resides in institutions or 
individuals is largely an artificial one that relates to how expertise is used 
rather than whether or not individuals can be experts. In this sense, an expert 
teacher in a dysfunctional school system would not be some kind of 
oxymoron, but rather a waste of human resources. Because larger cultural 
factors are central to both the development and practice of expert teachers, 
we would argue that expert teaching and its development require institutional 


supports and only make sense in a larger cultural context. 


Teaching Is a Cultural Activity 


Not only is teaching a complex system, but it is also a cultural system, what 


some have called a cultural activity (Gallimore, 1996; Stigler & Hiebert, 


1999). Our own thinking on this emerged from the Third International 
Mathematics and Science Study (TIMSS) video studies, conducted during the 
1990s (Stigler & Hiebert, 1999, 2004). In these studies, videotapes of 
national samples of eighth-grade mathematics and science lessons were 
collected in seven countries: Japan, Hong Kong, the Netherlands, 
Switzerland, Czech Republic, Australia, and the United States. The study had 
two goals: to document what “average” teaching looks like at a national level, 
and to compare teaching in the United States with teaching in countries that 
are high achieving when compared to the United States based on assessments 
of mathematics and science achievement. 

Findings from these studies indicated a striking homogeneity of teaching 
practices within countries, but marked differences in practices across 
countries. Even within a country as diverse as the United States — racially, 
ethnically, linguistically, and socio-economically — a national sample of 
eighth-grade mathematics teachers appeared to be following a common 
script, despite the fact that teachers are given high levels of autonomy and 
control over the methods they use. The US lesson script appears designed to 
produce what Skemp (1987) called an “instrumental” understanding of 
mathematics. Teachers walk through example problems, then supervise 
students as they practice solving similar problems, the goal being for students 
to remember the steps used by the teacher, and then to be able to execute the 


steps without errors. 


In Japan, by contrast, the lessons follow a different cultural pattern. 
Japanese teachers typically begin by presenting students with a difficult 
problem to work on, one they have not seen before. The teacher does not 
instruct students how to solve the problem, but simply lets the students 
struggle to find a way to solve it on their own. Because students have not 
been told how to solve the problem they usually come up with a variety of 
different solution methods — some correct, some incorrect — which they then 
discuss in class. Through these discussions, teachers focus on what Skemp 
would call a “relational” understanding of mathematics, working to deepen 
connections with core underlying mathematical ideas (see also Schwartz, 
Chase, Oppezzo, & Chin, 2011). 

Cultural routines such as these are not created deliberately. Instead, they 
evolve slowly over time as cultures adapt to an ever-changing environment. 
Modern schooling is itself a cultural invention, perhaps one of the most 
successful cultural inventions of our modern era. As the world economy 
shifted from agriculture to industrial production, schools developed to 
prepare workers who could fit into the jobs and living arrangements that were 
emerging. Schools, in a sense, went viral, and now they are everywhere. And 
significantly, the cultural routines of teaching that developed in the United 
States more than a century ago appear to be mostly unchanged (Cuban, 1990; 
Hoetker & Ahlbrand, 1969). 

Cultural activities are learned implicitly, which makes teaching quite 
different from most other domains of expertise. As a cultural activity, 
teaching is more like dinner-time conversation than it is like flying an 
airplane. The routines of dinner-time conversation are learned from growing 
up in a family and observing how others behave at meal times. People do not 


take a course or read a manual to learn this. They learn to participate in 


cultural activities by observing and imitating others. There is evidence that 
people learn to teach in just this way — by observing their teachers during 
their 13 years of schooling before entering college and by imitating what they 
remember (Lortie, 1975). 

Cultural activities can be hard to see simply because they are so widely 
shared within a cultural group. The teaching methods that are prevalent in US 
schools today appear natural to those of us who grew up here, but may appear 
strange to those who experienced a different tradition of schooling. Breaking 
free of our cultural lenses is one of the main benefits of cross-cultural 
comparison. When Japanese students are asked to solve problems they have 
never seen before, they struggle and often appear confused, yet teachers do 
not intervene to simplify the problems or resolve the confusion. When we 
observe this in Japan it suddenly draws attention to how uncomfortable US 
students and teachers are with the experience of confusion. These different 
cultural routines are supported by wider cultural beliefs about the role of 
confusion in learning. 

Finally, cultural activities are hard to change because they are multiply 
determined. Many factors conspire to keep things as they are. We know from 
cognitive science research that confusion is actually a critical part of deeper 
learning (D’Mello, Lehman, Pekrun, & Graesser, 2014). Yet if a US teacher 
tried to induce confusion in their students (let’s call it “productive 
confusion”) using teaching routines similar to those used in Japan, she would 
no doubt get a lot of pushback. Students would complain (“We haven’t had 
that!”), parents would complain (“It’s not fair to expect students to work on 
problems you haven’t taught them how to solve”), textbooks would not 


introduce material in the right order, and so on. The cultural nature of 


teaching raises problems for educational change, but it also has implications 


for the nature and development of teaching expertise. 


Expertise and Cultural Activities 


Cultural activities are implicitly learned and often operate outside of our 
awareness. This raises some interesting issues in terms of expertise. Most 
participants in cultural activities are neither experts nor novices; they are 
simply operating within the normal limits of variation for their cultural 


system. In Ericsson’s terms (Ericsson, 2008), they have reached a (perhaps 


premature) level of automaticity, a level at which they are “good enough.” 

An important consequence of the cultural nature of teaching is that one 
can be an expert in implementing teaching routines that are not themselves 
optimal. Hatano and Inagaki (1986) distinguished between two courses of 
expertise. Routine experts become efficient at implementing relatively set 
routines (an example might be someone working in a fast food restaurant) 
without necessarily understanding why they work or being able to reproduce 
them in a different setting. Adaptive experts (such as a sushi chef) deal with 
constantly changing problems and need to develop both an understanding of 
why things work as they do and an ability to alter their approach as 
circumstances change. 

It seems obvious that teaching involves adaptive expertise, because 
students present constantly evolving challenges to teachers. Yet to the extent 
teaching is nested within a set of cultural beliefs, the range of exploration of 
possible strategies will be limited. Those cultural beliefs vary across cultures 
but also change over time. Resnick and Resnick (1977) showed that societal 
expectations on literacy have changed dramatically over time. Strategies for 
teaching that work when only a low level of literacy is expected for most 


students will not be as successful when we expect much more from all 


students. This in turn has implications for what constitutes expertise in 
teaching literacy. 

Whether we should expect expert teachers to be able to transcend the 
limits of their cultural teaching routines is an open and challenging question. 
We have now discussed some of the obstacles to a straightforward analysis of 
expertise in teaching and described a model of what teaching entails. In the 
remainder of this chapter we will discuss what this means for studying and 


developing teaching expertise. 


Expertise in Teaching 


The first step in studies of expertise typically involves identifying a set of 
experts and analyzing how they differ from novices. In the case of teaching, 
three main approaches have been used, each with important limitations. 
These are: (1) comparing beginning and experienced teachers, (2) studying 
teachers who have been identified as experts through a process of nomination 
or certification, and (3) looking at student outcomes to identify expert 


teachers. 


Experience as a Proxy for Expertise 


A large body of literature supports the idea that expertise requires practice 
over long periods of time, but experience alone does not guarantee the 
development of expertise. Studies of the effect of teacher experience on 
student achievement generally find positive but small relationships between a 
teacher’s years of experience and their students’ learning (e.g. Nye, 


Konstantopoulos, & Hedges, 2004; Rockoff, 2004); some studies show 


benefits during the first several years, but none after that (e.g. Hanushek, 
2003; Rivkin, Hanushek, & Kain, 2005). The problem with conflating 
experience and expertise has long been recognized (e.g. Berliner, 1986). 
Despite this, experience still has been the main variable used to indicate 
expertise in teaching, at least until recently. Given that most practitioners in a 
domain will plateau before reaching the highest levels of expertise, these 
modest relationships probably underestimate the effects of expert teachers on 


their students’ learning. 


Studying What Recognized Experts Do 


The second approach to understanding expertise involves studying people 
who have been recognized as experts. One obstacle to this approach is that, at 
least in the United States, teaching is an activity that is often observed only 
by students. Berliner (1986) noted that judging of skill and outcomes in 
athletics, livestock, dogs, crops, and other fields takes years of systematic 
training and practice, but we lack anything similar in the evaluation of 
teaching. 

Efforts to systematically observe elementary and secondary school 
teaching have a long pedigree, going back at least to a program in the 1880s 
and 1890s that John Dewey participated in at the University of Michigan 
(Williams, 1998). More recently, the National Board for Professional 
Teaching Standards (NBPTS) has developed a process for certifying teachers. 
To date more than 112,000 teachers have completed their certification 
process, which involves submitting a multimedia teaching portfolio and 
taking a three-hour assessment examination. 

An early evaluation of the program (Bond, Smith, Baker, & Hattie, 
2000) looked at a set of 65 experienced teachers divided into groups based on 
their performance on the assessment, with approximately equal numbers 
passing and failing the NBPTS certification assessment. Interviews and 
classroom observations showed consistent differences favoring the 
designated expert teachers in, among other areas, the depth and challenge of 
problems set for students, the ability of teachers to anticipate and plan for 
classroom problems, and the depth of their representation of classroom 


situations. Student writing samples from the NBPTS certified teachers 


demonstrated higher understanding than did those from the comparison 
group. A later study by Hogan and Rabinowitz (2009) compared NBPTS 
certified and novice teachers and found similar differences in the depth of 
their representation of classroom problems. 

Efforts to show that NBPTS certified teachers have better student 
outcomes than do their non-designated peers have shown more mixed results. 
Cavalluzzo (2004) compared student achievement in mathematics in a large 
urban school district in classes taught by Board-certified and other teachers, 
finding consistent effects favoring the certified teachers. Goldhaber and 
Anthony (2007) and Clotfelter, Ladd, and Vigdor (2007) found that students 
of NBPTS certified elementary school teachers in North Carolina did better 
than those of uncertified teachers, with effects on the order of 0.05 standard 
deviations. Goldhaber and Anthony reported a larger effect, of 0.11 standard 
deviation for students eligible for free or reduced cost lunch. 

A much smaller effect was reported by Harris and Sass (2009), who 
looked at data from Florida. They also reported larger effects for teachers 
who had received NBPTS certification when it first became available. They 
argued that this suggests that the certification process initially identified a set 
of committed and effective teachers, but that as it became more widespread 
the certification process was less successful in identifying more effective 
teachers. 

The NBPTS has provided a systematic way of designating some 
teachers as at least relatively expert. There is some evidence that the teachers 
so designated differ from their peers in ways consistent with current ideas 
about effective teaching, but less consistent and convincing evidence that 


their students do better than do those of other teachers. This suggests that we 


might do better by simply identifying expert teachers based on how their 


students perform. 


Identifying Experts Based on Student Achievement 


An inductive approach to identifying experts involves looking at results and 
designating as experts those teachers whose students develop faster than 
expected. As discussed at the start of this chapter, this approach is at the heart 
of the “value added” approach to teacher assessment. Finding ways to 
connect teacher expertise to student outcomes will be critical to improving 
education, but efforts to do this to date have shown how complicated this 
seemingly straightforward approach is. 

One recent attempt to identify practices associated with student learning 
was the colossal Measures of Effective Teaching (MET) project sponsored by 
the Bill and Melinda Gates Foundation. The study was the most ambitious 
study of teaching ever undertaken: more than 20,000 videotaped lessons were 
collected from 3,000 elementary and middle-school teachers’ classrooms in 
seven urban school districts across the United States. Students were surveyed, 
and students’ test scores — both on state standardized tests and on 
supplemental tests of higher-order thinking in both mathematics and literacy 
— were collected and matched to the video data. Even more impressive, in the 
second year of the study a subsample of 800 teachers were randomly 
assigned, within schools, to different classrooms of students, and again, 
students’ learning at the end of the year was measured. 

This was the first large-scale study to identify teacher effects on 
students’ learning using random assignment. The measures of teacher 
effectiveness based on students’ learning in year one (non-random 
assignment) were highly predictive of the randomly assigned students’ 


learning at the end of year two (Kane, McCaffrey, Miller, & Staiger, 2013), 


and importantly, the size of the teacher effect in year one was the same as that 
in year two. Clearly, some teachers are more proficient than others at 
producing student gains on standardized state tests. But disappointingly, the 
observational measures applied to the videos of classroom teaching yielded 
very little of note, predicting almost none of the variance in student learning 
at the end of the year (Kane & Staiger, 2012). 

These findings are in some ways reminiscent of the early work on chess 
(de Groot, 1965, 1966). Despite numerous attempts to find differences in the 
way chess masters and weaker players play chess (e.g. number of moves 
considered, search heuristics, depth of search, etc.), the only reliable 
difference turned out to be in the quality of the move: chess masters make 
better moves in any given situation, and thus win more games, than the 
weaker players. 

In the case of teaching, we can imagine several additional reasons such 
correlations would be low. One might be the alignment between teachers’ 
goals and the measures used to indicate student outcomes. If standardized 
tests aren’t measuring the student learning outcomes of highest priority to 
teachers, then you would not expect a correlation between teachers’ 
behaviors in the classroom and students’ end of year test scores. But perhaps 
even more important is the contextual nature of teaching. A move that might 
be best in one situation might be precisely the wrong move to make in a 
different situation. For example, a critical remark to one student may be just 
what he needs to engage him in digging deeper on a problem. But the same 
remark may come as a crushing blow to a different student, who needs more 
encouragement. 

In this sense, teaching is more like driving to work than like shooting a 


rocket ship to the moon. Most of the work involved in shooting a rocket ship 


can be done in advance. The trajectory can be calculated with near-perfect 
accuracy, such that when the button is pushed, everything unfolds as 
predicted. Driving to work, on the other hand, requires constant adjustment to 
the expected and unexpected variations that occur. Taking a left at a certain 
intersection may usually be best, but not always. Unexpected obstacles or 
weather conditions require a response from the driver. The expert commuter 
knows when to adjust, when to go around. Teaching, thus, is more like 
driving to work. With goals in mind, teachers must constantly read the 
situation, monitor progress, and make necessary adjustments. 

This analysis leads us to reject the idea that expertise in teaching can be 
defined in terms of decontextualized “best practices.” Our view is that 
correlations between teacher actions and student learning are low not because 
we haven’t yet identified the right set of best practices, but because teaching 
itself is contextual, meaning that such correlations will always be low. 

Further support for this view comes from the TIMSS video studies. If 
expertise in teaching is defined by a set of best practices, then one would 
expect the practices used by teachers to be similar across the highest 
achieving countries. For example, because Japan is a high-achieving country 
in mathematics, we might expect that Japanese teaching routines (as 
described earlier) would be similar to those used in other high-achieving 
countries. 

In fact, however, this proved not to be the case. Although students in the 
Netherlands, Czech Republic, Switzerland, and Hong Kong — as well as 
Japan — all score relatively high on international mathematics tests, teaching 
methods across these countries vary markedly from one another. Features of 
teaching seen as desirable by education reformers in the United States — for 


example, the use of manipulatives, real-world problem scenarios, and group 


work — were found in some, but not all, of the high-achieving countries. 
Almost everything coded in the TIMSS video studies varied among the high- 
achieving countries (Stigler & Hiebert, 2004). 

We interpret this finding, again, as further evidence for the cultural and 
contextual nature of teaching. In Japan, a teacher can pose a difficult problem 
at the start of a lesson, and students will immediately set to work on it, even 
if they find it frustrating and uncomfortable. If an American teacher adopted 
this same practice the results might be quite different. In a project aimed at 
helping American teachers to increase student discussion in mathematics 
lessons (Wang, Miller, & Cortina, 2013), we found that helping teachers 
develop skills at leading mathematical discussions and giving them daily 
feedback was not sufficient to promote change. We had to include some 
“professional development” for students as well, because they did not know 
how to listen to and engage with each other’s mathematical ideas. Teaching 
expertise exists within a cultural matrix, which means that expert teachers in 


different cultures may act very differently. 


The Construct of Learning Opportunities 


If teacher expertise cannot be equated with a set of best practices, what does 
explain the considerable teacher-level variance in student learning outcomes? 
We believe a clue may be found by digging deeper in the TIMSS videos of 
classrooms in the high-achieving countries. In fact, we do see commonalities, 
not at the level of what teachers do but in the kinds of learning opportunities 
they manage to create and sustain for students. Teaching routines differ 
among the high-achieving countries. But although they use different routines, 
and the actions of teachers differ, all appear to have found ways to create a 
common set of learning opportunities for students. 

Based on results from the TIMSS video studies, and on our reading of 
research on teaching and learning more broadly, we propose three distinct 
types of learning opportunities that are necessary to produce high levels of 


learning in mathematics and, we believe, in other subjects as well: 


e Productive struggle — This can be simplified to the aphorism no pain 
no gain: deep learning requires some element of struggle. Despite the 
romantic view of learning as ideally fun and enjoyable, students learn 
more when they are engaged in hard intellectual work which, though 
rewarding in the end, is not necessarily enjoyable in the moment (see, 
for example, Bjork & Bjork, 2011). 


e Explicit connections — Evidence from the discovery learning literature 
suggests that just struggling with core content will not necessarily 
lead to learning (Kirschner, Sweller, & Clark, 2006). It is also 
important to help students make explicit connections between the 


problems they are working on and the concepts they need to 


understand. The source of these connections (individual discovery, 
listening to peers, or teacher summaries) may matter less than whether 


or not they are explicitly made. 


e Deliberate practice — Finally, struggle and connections need to be 
sustained over time, through practice; and not by repetitive practice 
but by deliberate practice, as demonstrated in the literature on 
expertise, with the opportunity to vary strategies and get informative 
feedback. 


Whereas the most common paradigm for researching the effects of 
teaching on learning looks for the simple effect of teacher actions/behaviors 
on student outcomes, we propose that a slightly more complex model will be 
required: teacher actions and behaviors create learning opportunities for 
students, which, in turn, produce student outcomes. Under this model, expert 
teachers are not defined as those who employ a set of best practices, but 
instead those who (1) have the ability to assess students’ current knowledge 
state both prior to and during instruction, (2) formulate clear learning goals, 
(3) consider a large number of strategies and routines in their repertoire, (4) 
make good judgments about which strategies are most appropriate in any 
given situation, and (5) are able to implement the strategies effectively to 
create learning opportunities for students. This model is represented in Figure 
24.1. 


Select and 
Implement 
Instructional 
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Progress 


Figure 24.1 A model of expertise in teaching. 


Are these additional layers really necessary? We believe they are. 
Besides the fact that simple correlations of teaching actions with student 
learning have proven less than fruitful, there is another reason: learning, 
especially the kind of deeper learning called for in current education 
standards, takes time. Unlike chess, where the final outcome can be known in 
the space of an hour or so, student learning of core content standards often 
takes place over months and years. Experts need to orient their actions 
simultaneously toward long-term learning outcomes and to more immediate 
indicators of learning. These more immediate indicators, we believe, are best 
found in a theoretically motivated construct such as learning opportunities — 


similar to what Lipsey (1993) has called “small theories.” 


Expertise in the Classroom 


What does it take for teachers to create these learning opportunities in the 
classroom, especially given the complex nature of teaching? First, teachers 
must have clearly defined goals for what they want students to learn, and the 
ability to assess the gap between students’ current knowledge and where they 
are trying to help students go next. Expert teaching is not just performing the 
acts of teaching. It is a highly contextualized endeavor in which teachers 
must create the precise learning opportunities that will move students to the 
next level of learning and development. Doing this requires great skill at 
formative assessment, which is one of the core elements of expertise in 
teaching (Black & Wiliam, 1998). 

Based on these assessments, which must be continually updated during 
instruction, teachers then must create learning opportunities targeted toward 
students’ current needs. Doing this will require knowledge, skill, and 


judgment. 


Knowledge 


Much research has focused on what expert teachers need to know in order to 
teach effectively. Clearly they need to know the subject that they are 
teaching. But this begs the question of what we mean by “know.” A great 
teacher of high school physics does not need to know physics as well as a 
professional physicist might know it. In fact, the professional physicist might 
be a below-average teacher of physics to high school students, or even to 
university students (see Feldon, 2007). Knowledge for teaching is not the 


same as knowledge for some other purpose. 


What do teachers need to know? Shulman (1987) coined the term 
pedagogical content knowledge to refer to the special knowledge teachers 
need in order to teach effectively. However, pedagogical content knowledge 
is just the tip of the iceberg. Shulman proposed a taxonomy of teacher 


knowledge that still proves useful today (Shulman, 1987, p. 8). It includes: 


e content knowledge; 


e general pedagogical knowledge, with special reference to those broad 
principles and strategies of classroom management and organization 


that appear to transcend subject matter; 


e curriculum knowledge, with particular grasp of the materials and 


programs that serve as “tools of the trade” for teachers; 


e pedagogical content knowledge, that special amalgam of content and 
pedagogy that is uniquely the province of teachers, their own special 


form of professional understanding; 
e knowledge of learners and their characteristics; 


e knowledge of educational contexts, ranging from the workings of the 
group or classroom, the governance and financing of school districts, 


to the character of communities and cultures; and 


e knowledge of educational ends, purposes, and values, and their 


philosophical and historical grounds. 


Much of the research that has been inspired by Shulman’s taxonomy has 
focused on pedagogical content knowledge. Hill, Ball, and colleagues (Ball et 
al., 2006; Ball, Thames, & Phelps, 2008; Hill & Ball, 2004) have invested 


considerable effort in further subdividing, and measuring, pedagogical 


content knowledge, developing measures referred to as the Mathematics 
Knowledge for Teaching (MKT) scales. 

Many researchers have studied the relationship between pedagogical 
content knowledge and student learning, but as with research on teacher 
practice, the correlations have been disappointingly low (Hill, Rowan, & 
Ball, 2005). One reason for these low correlations must certainly be the 
complex systemic nature of teaching; with so many variables in the mix, the 
correlation of any single one would be expected to be low. Another reason 
may be that the most common measures of pedagogical content knowledge 
consist of paper and pencil multiple-choice items, which may be measuring 
inert knowledge (Bransford, Goldman, & Vye, 1991; Whitehead, 1929). As 
with the case of teaching practices, the key point may not be whether or not 
you know something, but whether you are able to access and apply the 
knowledge when you need it to improve students’ learning opportunities. 

Kersting and colleagues have developed an innovative measure of 
teacher knowledge that they view as a measure of “usable” knowledge for 
teaching. This measure, called the Classroom Video Analysis (or CVA) 
assessment, requires teachers to view video clips of authentic classroom 
episodes and then comment on what they see in the video in terms of 
interactions among the teacher, students, and content being taught. Coding of 
teachers’ open responses yields a measure of teacher knowledge that 
correlates with Hill and Ball’s MKT measures (Kersting, Sherin, & Stigler, 
2014). More important, however: the CVA measures have been shown to be 
far better predictors of student learning than have the paper and pencil MKT 
measures (Kersting, Givvin, Thompson, Santagata, & Stigler, 2012; Kersting 
et al., 2016). 


Skill 


Despite the importance of teacher knowledge, there is a performance 
component as well to teaching. It’s not enough to just know what to do, but 
you also need to be able to do it well in a variety of situations. Most teacher 
education and professional development programs for teachers have focused 
on making teachers more knowledgeable; few actually give teachers 
opportunity to practice the skills of teaching. But some recent work has 
recognized the importance of skilled implementation for creating learning 
opportunities in classrooms. 

One such project is the one led by Lampert, Franke, and Kazemi 


(Lampert et al., 2013). These researchers have identified a set of instructional 


routines that are within the capabilities of pre-service teachers, yet still 
complex enough to represent the work of teaching. Such routines (which they 
refer to as activities) include Choral Counting, Launching and Using Word 
Problems, and so on. Participants in the program engage in “rehearsal” of 
these instructional routines through a process that very much resembles 
deliberate practice. 

Participants in the program begin by planning mini-lessons that 
incorporate one of the instructional routines, and then implement the lesson 
with their peers. After getting feedback from peers, they implement the 
lesson in a real classroom and record the result on video. They later take 
these videos back to the university and analyze their implementation, getting 
feedback from experts and peers. By using such routines as a site for 
deliberate practice, teachers in training are able to integrate the skills of 
teaching with the concepts and knowledge they are being taught in their 


teacher education program. The goal is to develop adaptive knowledge of the 


routines — the ability to implement them effectively to achieve instructional 
goals in a range of contexts. 

Instructional routines are not the only aspects of teaching that require 
skilled implementation. The skills of managing emotional connections with 
students are perhaps equally critical, especially if we expect students to 
engage in the hard work of productive struggle. Csikszentmihalyi, Rathunde, 
and Whalen (1997), in their influential study of talented teenagers, identified 
the relationship that students had with their early teachers or coaches as an 
important theme. Activities such as playing a violin or running long distances 
are not initially intrinsically rewarding, but having a teacher one does not 
want to let down can keep students engaged until they develop enough skill 
for the activity itself to be rewarding. 

This is an aspect of teaching that is often downplayed in US discussions. 
We interviewed elementary school teachers in China and the United States 
about their ideas of what contributed to their students’ success in learning 
mathematics (Correa, Perry, Sims, Miller, & Fang, 2008). Chinese teachers 
were much more likely than their US counterparts to emphasize the 
importance of their relationship with their students (as one teacher put it, 
“First of all ... I think I’ll let the students love me first. The students need to 
love their teacher before they love the subject, so I should develop a good 


relationship with them”). 


Judgment 


Finally, just knowing that and knowing how are not enough. Because 
teaching is highly contextual and complex, teachers must also have the ability 


to decide which of many possible strategies they should pursue in any given 


time and place. They need to be able to size up a situation, decide which 
strategy to employ, and then adapt it to achieve their specific instructional 
goal. In other words, teachers need judgment. They can be highly 
knowledgeable in all the ways Shulman (1987) describes, and highly skilled 
at implementing a wide variety of instructional strategies. But unless they 
make good decisions about when and how to employ their knowledge and 
skill, their knowledge may not serve to support students’ learning. 

Expert teachers’ judgments must be based, in part, on an expert reading 
of the situation, something that has been studied across a wide range of 
domains of expertise. Experts in fields from chess to radiology to electronic 
circuit design all appear highly skilled at extracting structure from the world 
as it relates to their domains of expertise (Hoffman, 1998; Kellman & 
Garrigan, 2009). Sometimes referred to as “situation awareness” (Endsley, 
1995), experts are able to quickly recognize the important features of 
complex situations and determine their consequences, whereas those who are 
less expert are more likely to miss key features, or spot them more slowly. 

These kinds of effects have also been found for teachers. Sabers, 
Cushing, and Berliner (1991) found that experienced teachers were better 
than novices at shifting their attention among multiple views of a classroom 
and identifying important events. In our lab, we have used mobile eye 
tracking methods to compare experienced and novice teachers teaching the 
same subjects to the same students (Miller, 2011). Experienced teachers are 
more focused in their attention to the important aspects of the classroom 
environment, such as students and curriculum materials, but some of the most 
interesting findings involve some of the trade-offs that beginning teachers 
need to make to tame the complexity of having to attend to a classroom of 


students at the same time they are trying to present a coherent lesson. A 


particular trade-off (Cortina, Miller, McKenzie, & Epstein, 2015) involves 
the balance of attention to individual students versus to the class as a whole. 
Beginning teachers who are rated as being highly responsive to individual 
students do not do a good job of dividing their attention evenly among the 
students as a whole, as shown by eye tracking data. Experienced teachers do 
not show this trade-off, with some teachers managing to be both responsive 
to individual students and attentive to the class as a whole. Situation 
awareness in teaching requires developing the kind of quick categorization of 
what’s important in a classroom situation, as measured by the Sabers et al. 
(1991) task and the measures developed by Kersting and colleagues 
(described above). To be an expert teacher, one must be able to perceive 
structure in educational contexts, then link that structure to underlying 
concepts and principles of teaching and learning and to the repertoire of 


strategies and routines that might be used to achieve the immediate goal. 


How Teachers Become Experts 


If there is a lesson to be learned from the huge literature on teacher 
professional development it is this: professional development, for the most 
part, doesn’t work. But then, most of what counts as_ professional 
development consists of haphazard, voluntary, and brief workshops that are 
disconnected from the daily work of teaching (Birman et al., 2007). Such 
activities may succeed in making teachers more knowledgeable, but because 
teaching is a complex cultural system, training just the teacher, in a time and 
place divorced from the ecology and culture in which they operate, is highly 
unlikely to improve the performance of the system as a whole. 

Indeed, a small number of carefully designed experimental studies show 
that when professional development is intensive, ongoing, and job-embedded, 
focused on students’ learning of the curriculum being taught, and aligned 
with the school’s improvement goals and priorities, such programs can 
produce significant improvements in student learning (Cohen & Hill, 2000; 


Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009; Saunders, 


Goldenberg, & Gallimore, 2009; Yoon, Duncan, Lee, Scarloss, & Shapley, 
2007). In other words, when other critical elements of the system of teaching 
are taken into account, and when teachers are given time to work on 
improving implementation within the context of the system in which they are 
working, teaching will improve. What does not work is training teachers 
apart from the setting in which they are expected to practice. 

There is a large literature on how to improve complex systems that 


include human actors, in fields ranging from automobile manufacturing to 


healthcare (e.g. Langley et al., 2009; Rother, 2009). We also know quite a bit 
about what it takes to change cultural routines (e.g. Gallimore, 1996; 
Feldman & Pentland, 2003), as well as what it takes to develop expertise 
across a wide variety of domains (Ericsson, Krampe, & Tesch-Rémer, 1993; 


Hatano & Inagaki, 1986). Rarely, until recently, however, have these 


Strategies and methods been applied to improving teaching (Bryk, Gomez, 
Grunow, and LeMahieu, 2015). In the next part of the chapter we hope to 


make some of these connections. 


Teachers versus Teaching as the Focus of Improvement 


Feldman and Pentland (2003), in their work on organizational routines, made 


a useful distinction. Routines, they write, consist of two related parts: “One 
part embodies the abstract idea of the routine (structure), while the other part 
consists of the actual performances of the routine by specific people, at 
specific times, in specific places (agency)” (p. 95). 

Teaching can be thought of in a similar way. The structure consists of 
the cultural routines, and the system in which the routines have evolved and 
function. Agency consists of the way the routines are implemented in 
different classrooms by different teachers. Improving teaching can thus be 
accomplished in two distinct ways: improving the system itself, including the 
routines, and improving the expertise of the teacher as she implements the 
cultural routines that she has inherited. 

As it turns out, the processes of improving the performance of complex 
systems, changing cultural routines, and developing expertise have a lot in 
common. In all cases, awareness is a critical part of the process — seeing 


clearly what the current system looks like. Improvement scientists (cf. 


Langley et al., 2009; Rother, 2009) talk about learning to “see the system,” 
which is never easy, but it is even more difficult when the system includes 
cultural routines. Building awareness of current routines (or skills in the case 
of expertise) is often better accomplished in collaboration with others — peers, 
team members, stakeholders, or even just a coach — because it requires that 
the routines be described explicitly — put into words. 

With a clear understanding of the system as it currently works (what 


Rother, 2009, calls the “current condition”), the next step is to establish a 


“target condition,” that is, what you want to see working differently (or more 
expertly) than it is now. Then, obstacles are identified — what is keeping you 
from reaching the target condition — and a next step is identified: what will 
you work on next to move the system incrementally closer to the next target 
condition? In systems improvement approaches such as the one described by 
Rother, this sets off a series of PDSA (Plan/Do/Study/Act) cycles, a process 
in which a change idea is planned and implemented, then analyzed to see 
what is learned. 

Importantly, developing the expertise of individual actors is not usually 
a goal of systems improvement. The goal, in fact, is usually just the opposite: 
to create a system that performs reliably at a high level without relying on 
high levels of expertise. If at all possible, the system should be designed so 
that all human actors, within a normal range of skill, can function effectively. 
According to Gawande (2007), this is the reason the field of obstetrics 
eventually gave up forceps in favor of the Caesarean section as a way to 
deliver babies in trouble: almost any physician could successfully execute a 
Caesarean section, whereas only those with high levels of expertise could use 
forceps effectively. When used expertly, forceps produce better results than a 
Caesarean. But when used by less skilled doctors, the results of a forceps 
delivery can be disastrous, and all obstetricians must begin as novices. 

The advantage of improving the system of teaching (as opposed to only 
the expertise of teachers) is obvious: changes in the system, even if small and 
incremental, can lead to long-term, large-scale improvements in student 
outcomes. Teachers come and go; most actually stay in the profession for 


only a few years (Simon & Johnson, 2015; Ingersoll & May, 2012). But 


system improvements that can be captured and accumulated over time have a 


far longer lifespan. Thus, it makes sense to shift at least part of our focus 


from teachers to teaching. But how can we accumulate system improvements 
so that they can be shared with each new generation of teachers? 

Morris and Hiebert (2011) put forth an interesting proposal, that 
instructional objects (e.g. curriculum materials, lesson plans, etc.) be seen not 
only as important components of an instructional system, but as a mechanism 
for storing professional knowledge. In their vision, educators will work 
together to continuously improve the system of teaching, capturing what they 
learn, as much as possible, in improvements to the actual instructional 
materials themselves. The shared instructional objects thus become a sort of 
knowledge base to guide teaching. 

Yet, despite the importance of improving the system of teaching — the 
curriculum, resources, methods, routines, and so on — there will always be a 
need, at the same time, to improve the expertise of teachers. As pointed out 
earlier, teaching is more like driving to work than it is like shooting a rocket 
to the moon: successful teaching will always include a large element of 
implementation. Because teaching is so highly contextual, and because so 
much judgment is required, it will always be necessary to develop individual 
teachers’ expertise to effectively implement routines for all students in all 
situations. 

Experience by itself will not lead to expertise. Instead, opportunities for 


deliberate practice will be required, to which we now turn. 


Creating the Conditions for Deliberate Practice of Teaching 


A large body of work has documented the critical role of deliberate practice 
in the development of adaptive expertise. Deliberate practice is not to be 
confused with repetitive practice. Anyone with lots of experience has also 
gained a lot of repetitive practice. But deliberate practice is something else. 
Deliberate practice is usually a designed experience — often by a teacher or 


coach — not one that happens naturally (Ericsson, 2006). It usually requires a 


set of training tasks that are outside the current comfort range of the learner, 
but not so far that they can’t be mastered in a matter of hours. 


According to Ericsson (2006), deliberate practice always involves 


conscious concentration on the skill, and informative feedback, both from the 
performance itself and from a coach or peers. This requirement for 
concentration “sets deliberate practice apart from both mindless, routine 
performance and playful engagement, as the latter two types of activities 
would, if anything, merely strengthen the current mediating cognitive 
mechanisms rather than modify them to allow increases in the level of 


performance” (Ericsson, 2006, p. 694). Hatano and Inagaki (1986) argue that 


the connection of concepts and understanding to the practice of routines is 
what makes the routines adaptive and able to be applied effectively in novel 
situations. 

There are a number of characteristics of teaching that work against 
successful implementation of deliberate practice. First among these might be 
the cultural assumption that teaching is not something generally subject to 
improvement. This belief might arise from a number of sources. Unlike other 


domains of expertise, the outcomes of teaching (e.g. student learning) are 


affected by numerous factors, the teacher being only one. Especially in 
environments where achievement is low, it may be difficult to convince 
teachers that the work they do can have a significant and direct impact on 
outcomes. If teachers don’t see their own skill as something that needs to be 
improved, it will be hard to engage them in deliberate practice. 

No matter how motivated teachers are to improve, there are other things 
that make it hard for them to engage in deliberate practice of teaching. Chief 
among these is the fact that, unlike many domains of expertise, teaching is 
one in which most of the teacher’s time is spent in performance, not in 
practice. Fadde and Klein (2010) point out that in music and sports, for 
example, performance happens rarely, leaving lots of time for practice in 
between performances. But many professions, for example, management, 
primarily consist of performance on a daily basis. In these kinds of 
professions — and teaching is clearly one — Fadde and Klein (2010) propose 
that deliberate performance, not deliberate practice, is the most feasible route 
to expertise. 

Fadde and Klein (2010) define deliberate performance as “the effort to 
increase domain expertise while engaged in routine work activity” (p. 6). 
Starting with features of deliberate practice (e.g. repetition, timely feedback, 
task variety, and progressive difficulty), they propose four types of exercises 
that professionals can engage in while on the job. These exercises include 
estimation (e.g. predicting how many students will use each of two different 
strategies for approaching a math problem), experimentation, extrapolation, 
and explanation. Fadde and Klein also propose that coaching is a much- 
needed resource for anyone embarking on a path of deliberate performance. 


Coaching would help to bring hidden routines to awareness, to make 


explanations explicit, and to provide feedback in domains where feedback is 
often difficult to come by. 

Thus, it is difficult but not impossible to create the conditions for 
deliberate practice of teaching. Our challenge is to create settings in which 
teachers can practice the implementation of a teaching routine or strategy; see 
the concrete results of their actions in evidence of students’ learning; reflect 
on the cause-effect relationships between their teaching, the learning 
opportunities they create for students, and the evidence of what students 
learn; and get feedback from knowledgeable others. We turn now to two 


examples of where this has happened. 


Lesson Study as a Lab for Deliberate Practice 


Much has been written about the Japanese practice of lesson study (Lewis, 
Perry, & Hurd, 2004; Lewis, Perry, & Murata, 2006; Stigler & Hiebert, 


1999). In lesson study, teachers meet regularly in groups of four to six to 


work on planning, implementing, and improving specific lessons from the 
curriculum. The goal of lesson study is not just to improve a particular lesson. 
Instead, the lesson becomes a vehicle for working on an improvement goal 
that goes well beyond a single lesson. So, for example, a lesson study group 
might be working on deepening students’ understanding of a particularly 
challenging concept, or on improving their ability to elicit students’ thinking 
through questioning. 

Similar practices are at work in other Asian countries, and the practice is 
also growing in popularity in the United States. Lesson study is relevant here 
for at least two reasons. First, it appears to have been inspired by the early 
work of Deming, a pioneer of improvement science who, coincidentally, 


spent much of his career working in Japan (Gabor, 1990; Kenney, 2008). As 


practiced in Japan, lesson study is first and foremost a research and 
development process, much like the PDSA cycles of improvement science 


(Langley et al., 2009). Second, lesson study appears to be aligned almost 


perfectly with the requirements for deliberate practice. In this sense it has the 
potential to greatly accelerate the development of expertise in teaching, as 
well as the system of teaching itself. 

The lesson functions similarly to the designed practice tasks described 
by Ericsson (2006). In lesson study, the practice of teaching is slowed down, 


often for weeks, as teachers discuss, first, what they want students to take 


from the lesson, and then what kinds of evidence they can collect to indicate 
students’ thinking and learning during the lesson. The group then plans, in 
great detail, a lesson that they hypothesize will achieve the goals they have 
set, and develops explicit hypotheses and predictions for both the processes 
and outcomes they will observe. Once they have created a detailed plan — a 
process that often takes place over several weeks — one of the teachers in the 
group will teach the lesson while the others observe, clipboards in hand, 
collecting evidence that then will be used to revise and improve subsequent 
versions of the lesson. 

Debriefing sessions follow the implementation of the lesson. In these 
sessions, teachers share the evidence they have collected, and analyze its 
significance for both understanding and improving the learning process that 
unfolds during the lesson. They focus special attention on places where their 
predictions were wrong, and seek to explain these deviations with 
cause-effect hypotheses. For example, they may observe some subset of 
students whose work indicates a misconception of the concept being taught. 
They then may think carefully about the task that was assigned, and even the 
specific questions the teacher asked as the work progressed, seeking to 
explain how the misconception could have emerged, and to find an 
incremental improvement that they can test in the next version of the lesson, 
which generally will be taught by a different member of the group. Through 
all of this the focus is kept on the lesson that the group has designed, not on 
the individual teacher who taught it, a strategy that helps to mitigate any 
defensiveness teachers may feel that could slow down their own learning. 

Ericsson has noted that premature automaticity may well be one of the 
biggest enemies to the development of expertise. Teaching, because of its 


heavy reliance on cultural routines, may be even more subject to this barrier 


to improvement than are other domains of expertise. Lesson study provides a 
means of disrupting the normal “good enough” routines of teaching. By 
spending weeks or months on a single lesson, each and every part of the 
lesson — both the parts that are planned ahead of time and the parts that are 
adapted at the time of implementation — is subject to deep analysis, revision, 
and practice, all in a context in which feedback is readily available. Lesson 


study is thus a context or lab in which deliberate practice can take place. 


Analysis as the Key to Developing Expertise in Teaching 


Even if lesson study can provide a setting for deliberate practice of teaching, 
it does not appear to provide enough opportunities for such practice to 
produce high levels of expertise over time. But what if the main things being 
practiced during lesson study are not the skills of teaching generally, but the 
specific skills teachers would need to learn from their own experience? In 
other words, perhaps lesson study is providing teachers with skills that can 
transform their daily work in the classroom into further opportunities for 
deliberate practice — something similar to Fadde and Klein’s (2010) theory of 
how deliberate performance might yield growth in expertise. 

A primary candidate for what these skills might be are analytic skills: 
teachers who participate in lesson study are learning to observe and analyze 
practice in ways that could directly improve their ability to read a classroom 
situation, select an appropriate strategy, and implement the strategy 
effectively, all while monitoring evidence of students’ thinking and learning. 


Specifically, teachers in lesson study are practicing: 


e Careful observation and analysis of students’ thinking and learning 


during a classroom lesson. 


e Generation of cause—effect theories (what Lipsey, 1993, calls “small 
theories’) that link a teacher’s actions to students’ thinking and 


learning during the lesson, and that may explain failures in learning. 


e Generation of alternative teaching strategies that may, if one’s theory 


is correct, lead to improved outcomes for students. 


e Testing of alternative strategies, and using what is learned to revise 


one’s own theories of classroom teaching and learning. 


If these specific skills can be applied by teachers to their own teaching, 
then daily experiences in the classroom can become a site for deliberate 
practice (or deliberate performance) and thus a possible mechanism for the 
continuous improvement of teaching over time. Teachers become more aware 
of their own teaching routines, and then develop and test changes in routines 
that might better address the needs of their students. Through iterative 
application of these skills, teachers will become better at reading situations, 
deciding which strategy to use, and adapting the strategy to meet their 
specific instructional goals. 

Evidence supporting this theory comes from work, reviewed above, by 
Kersting and colleagues in which teachers’ analyses of classroom video clips 
are used as indicators of pedagogical content knowledge (Kersting, Givvin, 
Sotelo, & Stigler, 2010). In this simple paradigm, teachers are asked to watch 
a series of short video clips depicting authentic classroom events, and are 
then asked to comment on the interactions among the teacher, students, and 
mathematics. Teachers’ analytic skills, measured in this way, have been 
shown to significantly predict students’ learning from pre- to post-test in a 
variety of mathematical domains, and the effect has been shown to be 
mediated by an observational measure of instructional quality (Kersting et al., 
2012). 

Although the relationship is correlational — i.e. nothing was manipulated 
in the study — other research lends further support to the idea that teachers’ 
analytic skills directly lead to improvements in instructional quality and 


student learning. Roth et al. (2011), for example, showed that teachers who 


participated in a year-long professional development program focused on the 
analysis of lesson videos (project STeLLa: Science Teachers Learning from 
Lesson Analysis) produced high quality instruction and higher levels of 
student learning than did a group of comparison teachers. And analysis of 


practice has become a key component of many teacher learning programs. 


Conclusion 


Teaching is a complex socio-cultural system, both hard to see and hard to 
change. Improving teaching requires that both the routines of teaching and 
the expertise of the teacher be improved. Although the aim of improving 
teaching routines is to enable an average teacher to produce the desired 
outcomes, the nature of teaching will always necessitate a certain level of 
adaptive expertise on the part of the teacher. Teachers must have knowledge 
(knowing that); they must have skill (knowing how); and they must have 
judgment — the ability to size up a situation, see its structure the way an 
expert physicist sees the structure of a physics problem, and then bring the 
right knowledge to bear so as to achieve the instructional goals. 

The models of teaching and of the development of expertise in teaching 
proposed here share the key feature that both require cycles involving 
analysis of a problem, planned activity, and assessment of the results that 
then informs future practice. This approach has proven successful in 
improving performance in a number of important domains, and we believe 
that it, rather than the promulgation of a decontextualized set of best 
practices, holds the most promise for improving the quality of teaching and 
learning. 

Perhaps because of the complexity and culturally nested nature of 
teaching, the fields of teacher education and teacher professional 
development have lagged in bringing principles of systems improvement and 
development of expertise to the task of improving teaching. Teaching 


presents a challenge for models of expertise — it is difficult to identify experts 


and the realities of teaching make it difficult to engage in deliberate practice. 
The concept of deliberate performance and work on improving performance 
in other complex fields suggest that there are promising ideas from other 
fields that can be usefully applied to understanding and improving the 


development of teachers. 
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Background and History of Naturalistic 
Decision Making 


Naturalistic Decision Making (NDM) research was initiated in the late 1980s 
as an exploration of how people make decisions in complex situations. The 
dominant paradigm, Judgment and Decision Making (JDM), did not seem 
applicable in many situations found outside the laboratory. Utility analysis of 
judgment accuracy and the emphasis on reasoning biases and limitations 


were normative (e.g. Dawes, 1998; Kahneman, Slovic, & Tversky, 1982) 


whereas early studies of expertise were aimed at developing descriptive 
models. Other approaches to decision making, such as the Brunswikian 


tradition (e.g. Hammond & Stewart, 2001) and information processing (e.g. 


Wickens & McCarley, 2008) were applicable to operational contexts, but did 
not focus on expert decision making in dynamic contexts — although it should 
be noted that more recent work from an information processing perspective 
has begun to examine expert—novice differences in attentional strategies (e.g. 
Schriver, Morrow, Wickens, & Talleur, 2008). 


NDM researchers promoted models more suitable to explain the 
reasoning of experts in domains characterized by dynamic conditions, time 
pressure, uncertainty, high stakes, multiple players, and organizational 


constraints (Orasanu & Connolly, 1993). Their studies suggested that 


decision making in these domains could not be reduced to a single moment of 
choice after all the facts had been analyzed. Rather, these situations often call 
for immediate action, with imperfect knowledge. Early NDM work (see Ross, 


Shafer, & Klein, 2006) revealed that domain experts rarely considered more 


than one option at a time, but instead expended most of their effort on 
situation assessment. Decisions emerge from recognition and/or making 
sense of the situation (Klein, Phillips, Rall, & Peluso, 2006b). 

Over the subsequent 25 years, NDM became a leading framework for 
the study of expert decision making in operational environments. The range 
of domains examined includes (but is not limited to) aviation (e.g. Cohen, 


Adelman, & Thompson, 2000), military activity (e.g. Cohen et al., 2000; 


Militello, Sushereba, Branlat, Bean, & Finomore, 2015), offshore oil 
production (e.g. Lauche & Bayerl, 2011), manned space operations (e.g. 


Fischer & Mosier, 2015), business management and leadership (e.g. Osland, 


2011), disaster response (Hutchins & Kendall, 2011), healthcare (e.g. Cesna 
& Mosier, 2005; Clark, 2013; Mazzocco et al., 2009; Patterson et al., 2016a), 
policing (e.g. Alison, Eyre, & Humann, 2011), professional sports (Johnston 


& Morrison, 2016; Macquet, 2009), weather forecasting (e.g. Hoffman, 


LaDue, Mogil, Roebber, & Trafton, 2017), and intelligence analysis (e.g. 


Rasmussen, Sieck, & Hoffman, 2013). Special issues in Cognition, 
Technology & Work (Vol. 17) and Journal of Cognitive Engineering and 
Decision Making (Vols. 10, 11, and 12) attest to the growth of NDM research 
over the past decades. In this chapter, we focus on expertise and decision 
making as defined by NDM theory, research, and practice. We discuss key 
articles defining NDM and demonstrating its applicability, new developments 
in theory and methods, work accomplished primarily since 2000, and 


challenges and future directions. 


The Concept of Expertise in NDM 


Expertise occupies a central place in NDM and is defined and evaluated 
differently than in prescriptive approaches, such as the Heuristics and Biases 
approach or Behavioral Decision Making (e.g. Kahneman et al., 1982). The 
“world” in NDM is unlike the probability-driven world that is captured in the 
laboratory. Consequently, NDM expertise is not evaluated according to 
Bayesian or probabilistic standards, but rather evaluation takes into account 
the complexity of the situation, the processes used, and the appropriateness of 
the course of action selected. Expertise is reflected in the breadth of domain- 
related experiences a decision-maker has, and the store of events he/she relies 
on as referents when assessing a situation in context. In addition, NDM 
studies of experts in diverse domains have uniformly demonstrated that 
individuals who achieve high proficiency engage not only in deliberate 
practice, but deliberate performance as well (Ericsson, 2006; Fadde & Klein, 
2010). 

While a focus of NDM is on experts, the literature on NDM is not 


unique in bifurcating people into experts versus novices. It needs to be 
understood, however, that these are two ends of a continuum, with apprentice 
and journeyman levels in between. Furthermore, junior and senior grades of 
proficiency can be identified within the main proficiency levels (Feltovich, 
Klein, Fiore, Ziebell, & Hoffman, 2009; Hoffman et al., 2014). NDM studies 


are also not unique in often relying on “years of experience” to decide 


whether an individual qualifies as an expert. Fortunately, these issues of 


definition and determination are becoming clarified and better understood 


within the NDM community, in part due to the impact of Ericsson’s 
insistence that expertise be operationally defined. 

NDM researchers look for individuals who have achieved a high level of 
skill in a particular domain, and then examine how these individuals develop 
and apply their expertise in context. Researchers have defined a number of 
overlapping variables of expertise that are important to NDM investigations 
(adapted from Phillips, Klein, & Sieck, 2004; Mosier & Fischer, 2010), 


including the following. 


Perceptual Learning and Skills 


As can be seen in a number of chapters in this volume (see Norman et al., 
Chapter 19; Durso, Dattel, & Pop, Chapter 20; Cross, Chapter 21; Williams, 
Ford, Hodges, & Ward, Chapter 34), perceptual discrimination and pattern 
perception are skills that have to be learned in the course of achieving 


expertise in many domains (for reviews, see Fahle & Poggio, 2002; Hagen & 


Tanaka, 2014). Perceptual learning does not happen by mere exposure to 
exemplars but is acquired through deliberate acts of discrimination and 
differentiation combined with corrective feedback. Studies of domain experts 
have made it clear that experts learn to perceive patterns that non-experts do 


not discern (e.g. Hoffman et al., 2014; Lesgold, Rubinson, Feltovich, Glaser, 


& Klopfer, 1988), and to discriminate among stimuli (e.g. Weiss & Shanteau, 


2003). They are able to quickly identify the most ecologically valid cues, that 
is, the subset of information most critical to accurate situation assessment, 
and to match patterns of cues against patterns in their experience base. 
Patterns are configurations of cues, including cues that are absent (Goldstone, 
2000). In addition, cues are often dynamic, and sometimes the patterns can 
only be defined across multiple data types; for example, in weather 
forecasting, patterns indicative of the formation of severe storms are defined 
in terms of changes observed across the sweeps of the radar, the data on the 
directions of winds at various heights in the atmosphere, and other 


measurable dynamic atmospheric parameters (Hoffman et al., 2017). These 


perceptual skills provide a critical edge in time-constrained situations. 


Mental Models/Simulation 


Experts have cognitive structures — mental models — that represent how 
things work in their domain of practice and allow them to rapidly understand 


situations (Ross, Battaglia, Phillips, Domeshek, & Lussier, 2003) and to 


predict developments. Experts are sensitive to changing conditions and adapt 


their mental models accordingly (Waag & Bell, 1997). Mental simulation is 


the process of consciously envisioning a sequence of events. Experts run 
mental simulations to understand how a situation might have evolved to the 
point at which they found it, or to refine their action choice if they discover 


potential problems during the mental simulation (Klein, 2008). 


Sense of Typicality and Associations 


Experienced decision-makers have accumulated a vast repertoire of cases to 
draw upon (e.g. Chi, Glaser, & Farr, 1988; Klein, 1993). They recognize 


what is typical in a situation (Ericsson & Simon, 1993), and they recognize 


when things are not going as expected — when there is an anomaly or 


something is missing. 


Routines 


Expert decision-makers possess a wide repertoire of tactics to respond 
quickly and adaptively to familiar situations. Often, decisions are made 
incrementally and iteratively as experts use feedback from the environment to 
adjust their actions when a decision is not working effectively (Lipshitz & 
Strauss, 1997). 


Declarative Knowledge 


Experts know more domain-related facts and details than non-experts. 
Charness and Tuffiash (2008) noted that experts’ superior performance in 
domains such as sports, medicine, and aviation is directly dependent on their 
domain-specific knowledge, and that this knowledge enables experts to 


anticipate future actions and prepare for them more efficiently. 


Tacit Knowledge 


Experts may operate without conscious effort, and the tacit knowledge 
supporting expertise is neither verbally encoded nor easily articulated 
(Crandall, Kyne, Militello, & Klein, 1992). The concept of tacit knowledge 
was introduced by Polanyi (1958, 1966), to explain why “we know more than 


we can tell” (1966, p. 4). It refers to the things we know but cannot put into 


words. Reber (1993) empirically demonstrated the phenomenon of implicit 
learning using an artificial grammar; participants were successful at assessing 
the “grammaticality” of letter strings even though they were unable to 
articulate underlying rules. 

Polanyi’s concept of tacit knowledge built on the ideas of Ryle (1949), 
who had distinguished knowing that (declarative knowledge) and knowing 
how (procedural knowledge). Klein (2009) suggested that this distinction 
fails to convey experts’ perceptual skills, and proposed additional features of 
tacit knowledge: perceptual recognition, pattern matching, recognition of 
typicality and anomaly, and mental models. None of these is easily 


explicated, but all seem essential to expertise. 


Situation Assessment 


Experts’ ability to use pattern-matching facilitates their situation assessment 
as they are able to comprehend quickly what may be at the core of a problem 
and what actions should be taken (Orasanu & Connolly, 1993). In cases that 
are not amenable to pattern matching, processes such as sense-making 
(discussed below) may be used to reach an understanding of the situation or 
problem. Situation assessment is also characterized by action/feedback loops, 
an iterative process to create updated models of changing situations and 
incorporate changes in future projections that result from incremental 
decisions (Endsley & Jones, 2012). 


Finding Leverage Points 


Leverage points are opportunities for making critical changes with relatively 
low effort. For example, to manage time wisely, experts anticipate 
developments, make contingency plans, prioritize tasks, and use low- 
workload periods to prepare for upcoming events (Fischer, Orasanu, & 
Montalvo, 1993). 


Managing Uncertainty 


Experts have a range of strategies for managing uncertainty, ambiguity, and 


time pressure in their work domain, including information seeking and time 


management (Cohen, 2011; Fiore, Rosen, & Salas, 2011). Experts are 
proactive, anticipate potential failures, risks, or conflicts, and prepare for 
them. For instance, wildland firefighters use a two-pronged approach, 
combining anomalizing (discerning discrepancies) with proactive leader 
sense-making to develop a greater understanding of the situation (Barton, 
Sutcliffe, Vogus, & DeWitt, 2015). 


Metacognition 


Experts not only monitor what but also how they are thinking and whether it 
is appropriate for the situation at hand. They critique and correct their 
diagnosis until they arrive at a satisfactory mental model of the situation, or 


further processing is too costly (Cohen, 2011; Khoo & Mosier, 2008). They 


are able to shift strategies when faced with high uncertainty or unmet 
expectancies, taking an incremental approach or engaging in more analytical 
processes (Klein, 2003; Moulton, Regehr, Mylopoulos, & MacRae, 2007). 


For example, expert surgeons perform many routine tasks automatically, but 


‘slow down’ and engage in effortful processing in preparation for non-routine 
events or in response to unexpected events (Moulton, Regehr, Lingard, 
Merritt, & MacRae, 2010). 


NDM Frameworks 


NDM frameworks incorporate the variables of expertise described above. 
Importantly, they emphasize and unpack the “front end” of decision making — 
the diagnosis or situation assessment phase (Mosier & Fischer, 2010), 
another facet that distinguishes NDM from traditional decision making 
approaches. Below we describe the most commonly used frameworks and 


expert strategies that have emerged from NDM research. 


Recognition-Primed Decision Making (RPD) 


Klein’s RPD model, perhaps the most well-known NDM model, was derived 


from interviews with fireground commanders (FGCs; Klein, Calderwood, & 


Clinton-Cirocco, 1986). “Critical incident” descriptions of non-routine or 
command-challenging situations indicated that approximately 80 percent of 
FGC decisions were recognition-based rather than analytical, and depended 
on their skill at recognizing situations as typical, understanding of the causal 
dynamics at work, and identifying promising courses of action. Commanders 
then used serial evaluation via mental simulation to examine options. 

The RPD model includes a blend of intuition and analysis (Klein, 2008). 


It posits that experts in high-stakes, time-pressured decisions rely on their 


experience and pattern-match when confronted with a situation recognized as 
familiar or typical, characterized by cues, expectancies, suitable goals, and 
typical courses of action (COAs). Recognition of situation typicality often 
leads directly to action and involves no comparison of options, because the 
situation is linked to a COA that the expert already knows will work (e.g. 
Johnson & Raab, 2003). Analytical processes may be involved in situation 
assessment when situations are not familiar to experts or when facets of the 
situation do not meet expectancies. These strategies may involve “sense- 
making” — situational elements are related in a causal and/or temporal order 
to account for unexpected or inconsistent elements (Weick, 1995; Klein, 


Moon, & Hoffman, 2006a) — or evaluation of a course of action via mental 


simulation (Klein, 2008) to enable an understanding of the situation. 


Key studies discussed in the previous edition of this handbook 


contributed to the development and refinement of the RPD model. An 


interesting new application of the RPD framework is to the domain of sports 


(e.g. Macquet’s analysis of expert volleyball players, 2009). 


Dealing with Uncertainty 


Uncertainty and ambiguity are hallmarks of NDM domains and situations. 
Although the terms are often considered to be equivalent, Carroll (2015) 
characterizes ambiguity as the general frame for uncertainty — “not knowing 


for sure” (Grote, 2014). Several NDM researchers have proposed frameworks 


for dealing with uncertainty (see preceding sections on Managing Uncertainty 


and Metacognition). 


Recognition/Metacognition (R/M) and RAWFS 


R/M (Cohen, 2011) with its associated STEP procedure (construct a Story, 
Test its plausibility, Evaluate its goodness, Plan for the possibility of error) 
focuses on situations in which recognition does not work. STEP is a process 
of constructing causal models or stories, which expand and change as experts 
collect and synthesize data. The STEP procedure was successfully used in 
training novices to display target behaviors and decisions similar to those of 
experts (Cohen, 2011). 

The RAWES heuristic (Lipshitz, Klein, Orasanu, & Salas, 2001) is 
another tactic to cope with three principal forms of uncertainty: inadequate 
understanding of the situation, lack of information, and conflicted 
alternatives. Steps in RAWFS are Reducing uncertainty, Assumption-based 
reasoning, Weighing pros/cons of alternatives, Forestalling, and Suppressing 
uncertainty. RAWFS has recently been applied to the study of paramedics in 
non-routine situations, and results indicated that forms of uncertainty 


influenced coping strategies (Haren¢arova, 2017). 


Capturing and Unpacking Expertise 


Since 2000, NDM and associated frameworks described above have matured, 
established methods have been refined, and new tools, models, and methods 
have been developed, particularly in regard to making explicit experts’ tacit 
knowledge for training purposes. 

Cognitive Task Analysis (CTA; Hoffman & Militello, 2008) and 
Applied Cognitive Task Analysis (ACTA; Millitello, Wong, Kirschenbaum, 
& Patterson, 2011), designed to capture tacit cognition, strategies, and 
knowledge of experts in context, are widely used in NDM efforts (see also 
Lintern, Moon, Klein, & Hoffman, Chapter 11, this volume). CTA methods 
include the Critical Decision Method (CDM) and the Knowledge Audit (KA). 
In CDM, experts are asked to recall a critical incident, and to think through 
the incident several times, unpacking more details with each sweep (e.g. 
Patterson et al., 2016a). KA complements CDM, as questions probe aspects 
of expertise such as diagnosis and prediction, improvisation, metacognition, 
anomaly detection, and compensating for technology limitations. Often KA is 
used early in a study to obtain an overview of the knowledge and skills 
needed in a specific domain; it has also been used to elicit knowledge needed 
for training (Gore, Banks, & McDowall, 2015). 

CTA is often the first step in creating programs for training expertise. 
For instance, Zimmerman, Sestokas, and Bums (2011) employed CTA to 
identify decision points that were then incorporated in training scenarios for 
the Joint Forces Air Component Commanders. Likewise, Johnston and 


Morrison (2016) designed training programs for rugby playmakers based on a 


CTA analysis of professional players. Gore et al. (2015) used ACTA with 
expert engineers to identify task elements that were the most cognitively 
complex with respect to judgment and decision making. They then probed 
those elements in a KA to get at the engineers’ tacit knowledge of key cues 
and strategies, and created a computer-based training program for novice 
engineers. In the medical domain, Patterson et al. (2016a) used CDM with 
residents and expert faculty to elicit knowledge supporting a simulation- 
based training for early recognition of sepsis. 

CDM may also be used in combination with other forms of data 
collection. Roberts, Flin, and Cleland (2015), for example, employed CDM, 
observations, and video analysis to identify underlying cognitive components 
associated with the development and maintenance of situation awareness in 
offshore drillers. Extensions of CTA to the organizational level include 
Applied Concept Mapping (Moon, Hoffman, Eskridge, & Coffey, 2011) and 
NDM-based Expertise Management. The latter aims to externalize and 
preserve expertise to ensure consistency in performance when others take on 


the expert’s responsibilities (Moon, Baxter, & Klein, 2015). 


Expert Teams 


Initially NDM _ research focused on the individual expert while 
acknowledging that decisions in many operational settings involve “multiple 
players” (Orasanu & Connolly, 1993, p. 10). Over the past two decades, 
studies on team cognition have gained in prominence as evidenced by an 
increasing number of chapters on this topic in NDM conference proceedings 
(e.g. Hoffman, 2007; Mosier & Fischer, 2011; Salas & Klein, 2001). The 


overarching goal has been to identify expert team processes and knowledge 


supportive of team performance by co-located teams — e.g. flight deck crews 
(Bourgeon, Valot, & Navarro, 2013), air traffic controllers (Smith-Jentsch, 
Mathieu, & Kraiger, 2005), and medical teams (Nemeth, 2008) — as well as 


by teams that involve remote collaboration, for instance, space crew members 
and mission controllers (Bearman, Paletz, Orasanu, & Thomas, 2010; 
Fischer, Mosier, & Orasanu, 2013), pilots and air traffic controllers (Mosier 
et al., 2013), onshore/offshore oil drilling teams (Lauche & Bayerl, 2011), 


teams involved in air traffic management (Smith, Spencer, & Billings, 2011), 


disaster management (Smith & Dowell, 2000), maritime interdiction 
operations, air warfare decision making, and emergency response (Hutchins 
& Kendall, 2011). 

When decision making is the result of a team effort, communication and 
shared mental models are crucial. Shared mental models represent team 
members’ knowledge about their operational environment, their task 
(especially mission objectives), teamwork (e.g. each member’s 
responsibilities, capabilities, and information needs), and their current 


situation. They provide team members with an interpretive framework for 


joint actions, facilitate communication, and enable them to respond 
adaptively to changing task conditions (Salas, Cooke, & Rosen, 2008). 
Traditionally the notion of “shared” has been equated with knowledge 
convergence. An alternative conceptualization, related to transactive memory 
(DeChurch & Mesmer-Magnus, 2010), emphasizes the complementary nature 
of team members’ knowledge and takes the word “shared” to mean 
“distributed” (Mohammed & Dumville, 2001). There is empirical evidence 
for both views (Smith-Jentsch, Cannon-Bowers, Tannenbaum, & Salas, 2008; 


Sperling & Pritchett, 2005) suggesting that the benefits of overlapping and 


complementary knowledge concern different types of knowledge and team 


compositions (Reader, Flin, Mearns, & Cuthbertson, 2011). For instance, 


while team members ought to pursue the same mission goals and have a 
common understanding of their roles and responsibilities, different task 
knowledge is advantageous when tasks require the pooling of diverse 
expertise or division of labor (Orasanu, 2005). An issue that concerns both 
conceptualizations of shared mental models is the extent to which members’ 
knowledge accurately reflects task and teamwork requirements. Recent 
analyses show that mental model similarity and accuracy jointly predict task 
performance (e.g. Burtscher, Kolbe, Wacker, & Manser, 2011; Cooke, 
Kiekel, & Helm, 2001). 

As they operate in dynamic task environments, expert team members 
need to respond adaptively to changing conditions. Communication plays a 
pivotal role in this process (Orasanu & Fischer, 1992). Communication 
features associated with expert team decision making and performance 
concern inclusive interactions among team members. In decentralized teams, 
members contribute equally and interact directly with each other (Fischer, 


McDonnell, & Orasanu, 2007). In centralized teams, the leader serves as a 


hub receiving and disseminating task-critical information to the rest of the 
team (Kiekel, Cooke, Foltz, Gorman, & Martin, 2002). Expert teams also 
tend to discuss the problem they face and responses to it in great detail 
(Mazzocco et al., 2009; Tschan et al., 2009). Team members let others in on 
their reasoning and inform them about their intentions and expectations 
(Manser, Foster, Flin, & Patey, 2013). They ensure common ground by 
providing feedback (Johannesen, 2008), and mitigate errors through team- 
centered communication (Fischer & Orasanu, 2000). 

Remote collaboration tends to be cognitively more demanding compared 
to collocated teams (Olson & Olson, 2006) as team members lack a shared 
visual field and access to non-verbal cues. Communication protocols as well 
as technological solutions (e.g. video links; shared whiteboards) have been 
developed to support communication in remote teams (Fischer & Mosier, 
2015; Bayerl & Lauche, 2011). 


Training for Expertise 


One of the primary goals of characterizing expertise is to enable the 
development of expertise in others. A number of NDM-based training 


approaches have proven effective in pursuit of this goal. 


Decision Skills Training 


Decision Skills Training (DST) aims to accelerate the transition toward 
expertise by targeting the judgments and actions required by practitioners. 
DST is based on an understanding of the strategies that experts use to fine- 
tune their skills to dynamically changing conditions. These strategies include: 
being mindful of opportunities for learning; engaging in deliberate practice 
that incorporates goals and evaluation criteria; honing metacognitive skills; 
seeking feedback that is accurate and diagnostic; compiling an extensive store 
of cases; and building mental models (Pliske, McCloskey, & Klein, 2001). 

The training uses low-fidelity simulation exercises constructed from 
CTA data and presents lived experiences in terms of cues, strategies, factors, 
and novice difficulties. In working through the scenarios novices may engage 
in important functions such as exploring and revealing the limits of mental 
models and pattern matching, receiving targeted performance feedback, and 
participating in structured discussions on process accuracy (Pliske et al., 
2001; Ross, Lussier, & Klein, 2005). 


ShadowBox™: A Cognitive Tool 


ShadowBox is a scenario-based training approach that enables trainees to 
think like an expert (Hintze, 2008; Klein & Borders, 2016). Trainees are 


presented with realistic scenarios that involve off-nominal events as well as a 


list of possible responses — i.e. actions, goals, or priorities. Trainees rank 
options, explain their rankings, and compare their solutions with those of a 
panel of subject-matter experts. Trainees can thus experience the scenarios 
through the eyes of the experts, without having any experts present. Several 
studies demonstrated the training effectiveness of ShadowBox, after only a 
few hours of training (for a review, see Klein & Borders, 2016). 

ShadowBox overcomes the difficulty of getting access to experts during 
training sessions. Further, the materials can be presented as text or as video 


scenarios, or via computer to support distance learning. An expansion of the 


ShadowBox approach (Klein, Hintze, & Saab, 2013) targets other decision 
aspects: cues to monitor, issues to worry about, or interpretations of 
ambiguous situations. ShadowBox can also be used during knowledge 


elicitation with experts, or to evaluate personnel with the match to experts 


serving as a basis for assessment (Klein & Borders, 2016). Additionally, 
leaders can substitute their rankings and rationale for those of the experts, so 
that their subordinates are able to predict how their leader views difficult 


situations. 


Transfer of training 


While NDM emphasizes the domain specificity of decision making skill and 


the importance of experience in actual situations, it does not call for “high 


fidelity” of training scenarios. Instead, NDM training studies typically rely on 
low-fidelity approaches such as decision making exercises or ShadowBox 
scenarios. What matters in training is cognitive rather than physical fidelity 
per se (Hoffman et al., 2014). Thus, the low-physical-fidelity simulations 
used by NDM practitioners stress ingredients of cognitive fidelity: the types 
of decision strategies, the degree of ambiguity and uncertainty, the trade-off 
between goals, the clarification and even replacement of ill-defined goals, 
and the early attention to subtle cues. 

An important challenge is to determine whether this kind of simulation 
training transfers to decision making in operational settings even though 
characteristics of these settings make it difficult to establish a clear linkage. 
Nevertheless, several studies suggest that the cognitive fidelity employed 
during training does improve trainees’ decision performance in the field. For 
instance, training based on CTA resulted in reduced time to complete surgical 
procedures and decreased rate of errors (Clark, 2013; Sullivan et al., 2008). 
Similar results were found with NDM training of submarine technicians 
(Kirschenbaum, McInnis, & Correll, 2009). Likewise, NDM tactics were 
shown to improve performance in sports (Fadde, 2007; Ward, Suss, & 
Basevitch, 2009). 


Macrocognition: Extending the Focus of NDM 


Advances in unpacking and training expertise have led to perhaps the most 
significant development in NDM theory in the past decade — a recognition 
that decision making in naturalistic contexts is not limited to the intuitive, 
System 1 (Kahneman, 2011), pattem-matching processes, and System 2 
mental simulation described in the RPD model. Rather, NDM includes a 
variety of high-level cognitive functions, such as sense-making or knowledge 
construction, team collaboration and coordination, adaptation and replanning, 
that are typically present in “natural” (versus artificial laboratory) decision 
making settings (Klein et al., 2003; Schraagen, Klein, & Hoffman, 2008). 
Consequently, the term macrocognition has been adopted as a frame for 
NDM work (see Klein et al., 2006b). Concepts related to macrocognition, 
such as “situated cognition” (Brown, Collins, & Duguid, 1989) and 
“extended cognition” (Hollnagel, 2001) reference the fact that 
macrocognitive functions are generally performed as collaborations, in the 
context of a team working in a natural situation, and usually in conjunction 
with computational artifacts. 

Traditionally, cognitive researchers have conducted laboratory 
experiments using narrowly defined tasks such as puzzles, random visual 
patterns, or word lists and measuring “microcognitive” functions such as 
reaction time, serial versus parallel attention mechanisms, strategies involved 
in puzzle solving, or deductive reasoning to identify the building blocks of 
cognition. In contrast, macrocognition “only rarely looks at phenomena that 


take place exclusively within the human mind or without overt interaction. It 


is .... more concerned with human performance under actual working 
conditions than with controlled experiments” (Cacciabue & Hollnagel, 1995, 
pp. 57-58). NDM researchers conceive of their approach as_ being 
complementary to the microcognitive approach, as it is possible to 
incorporate the control and manipulation of cognitive variables into field 
research (Hoffman & Deffenbacher, 2011). 

The effects of context and the important role of sense-making in real- 
world situations mean that the analysis of expert decision making in any 
given domain will likely require multiple kinds of models to integrate 
macrocognitive functions and processes into NDM frameworks. Two 
important models are the Data/Frame Model of Sense-making, and the 


Flexecution Model of (Re-)planning. 


The Data/Frame Model of Sense-Making 


The Data/Frame (D/F) model emerged from studies of military decision 
making (Klein, Woods, Bradshaw, Hoffman, & Feltovich, 2004) and 
included cases of sense-making that showed that reasoning involved not only 
problem-solving, as traditionally conceived, but also detecting problems, 
apprehending relationships among complex and dynamic variables, forming 
potential explanations, thinking ahead, and identifying leverage points. The 
D/F model shown in Figure 25.1 depicts what happens when decision-makers 
try to understand complex situations, and continually work to refine and 


improve that understanding. 
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Figure 25.1 Data/frame model of sense-making. 


The processes within the D/F model are regarded as parallel and 
interacting. When one looks retrospectively at an instance of reasoning, one 


can describe it as a causal chain: (1) an initial frame is formed, (2) it is used 


to seek confirming and disconfirming evidence, (3) the frame is elaborated, 
or discarded, etc. This is essentially the model of problem-solving presented 
by Duncker (1945). The D/F frames have the qualities of causal networks of 
interacting events and forces. They are a form of abductive inference, that is, 
the inference of a best (albeit tentative) explanation (see Fann, 1970). 
Although we can retrospectively describe instances in which reasoning 
seems to follow a sequence with apparent start and stop points (a decision is 
“made”), reasoning “in the moment” consists entirely of closed loops as in 
Figure 25.1 (see Hoffman & Yates, 2005). Sense-making often proceeds in 
fits and starts, with gaps, distractions, and multitasking. Beginnings and 
endings can be anything but clear-cut. Thus, there are no input and output 
arrows. The sense-making process can commence anywhere, though it is 
often triggered by surprise (for examples see Hoffman & Militello, 2008). As 
decision-makers also take actions — they plan, and most importantly, they re- 
plan — a complete model of macrocognitive work must also include a model 


of activity. 


The Flexecution Model of (Re-)planning 


The Flexecution model combines concepts in macrocognition with concepts 


from conflict management (Klein & Rothman, 2008). A traditional process of 


“management by objectives” starts by defining the goal and then preparing a 
plan to achieve it. However, it falls short of describing how people plan when 
faced with ill-defined goals and difficult problems. This involves something 
more like management by discovery, as the initial goal is revised and the 
planner learns more about the domain and situational constraints and 
opportunities. The Flexecution model seeks to capture what successful 
problem-solvers do when adapting their goals in complex situations (Klein, 
2007a, 2007b). 

Traditional conceptualizations assume that a plan is made and then 
followed lock-step. In contrast, the Flexecution model of (re-)planning 
(Figure 25.2) recognizes that in complex reasoning and problem-solving, 
people clarify and adapt their goals while pursuing them, and that there are 
often no clear-cut starting or halting points in reasoning about complex 


causation (Hoffman, Klein, & Miller, 2011). Thus, it treats the goals of 


planning not as static input but as both inputs and outputs of the planning 


process. This distinction, while subtle, is also fundamental. 
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Figure 25.2 Flexecution model of (re-)planning. 


The closed loops in the Flexecution model are counterparts to those in 
the D/F model. Consistent with our understanding of human planning and re- 
planning, the Flexecution model does not assume fixed “start” and “stop” 
points. 


The data that support these models stem from diverse, cognitively 


complex environments. Osland (2011), for example, described the cognitive 
tasks of global organization leaders in terms of sophisticated sense-making 
frames that reflect the characteristics of the global context —complexity, 
multiplicity, interdependence, ambiguity, flux, and cultural variations. 
Stewart, Dominguez, and Way (2011) used the D/F model to analyze reports 
of surgeons after performing laparoscopic cholecystectomies in which major 
bile duct injuries had occurred. They found that in 60 percent of the cases, 
surgeons held on to the “normal surgery” frame and continued the procedure. 
Only when the surgeons questioned the original frame were bile duct injuries 


considered and mended. 


Emerging Challenges and Future Research 


While NDM_ research has made considerable contributions to our 
understanding of expert decision making in “real-world” contexts, important 
issues have not yet been sufficiently addressed. These include the process of 
expertise development, the role of affect in expert decision making, and the 
impact of culture on NDM processes such as sense-making. Technology such 
as simulators and eye-trackers provides new opportunities to study and train 
expert performance, and should further our understanding of expertise and its 


development. 


Expanding the Study of Domain Expertise 


NDM has historically focused on the study of individuals who possess expert 
level knowledge and skill as the method of providing benchmarks for expert 
performance. However, one cannot fully understand the endpoint (expert 
knowledge and skill) without knowing the progression from novice to 
apprentice, apprentice to journeyman, and journeyman to expert. Methods of 
accelerated learning to expertise have been proposed (Feltovich et al., 2009), 
and there is evidence of accelerated expertise development in highly 
motivated individuals through (a) training of skills identified as critical to the 
task or mission, (b) constant “stretching” of skills through challenging cases, 
(c) practice based on mentoring or expert instructional guidance, and (d) 
provision of rich meaningful feedback (reviewed in Hoffman et al., 2014). 
Deliberate practice is working hard, at hard problems. Accelerated learning is 
the notion of speeding up the achievement of proficiency and shortening the 
time it takes to achieve a given level of proficiency. Acceleration may 
involve shortening the time taken to move from apprentice to journeyman, 
and not merely the time taken to get someone from novice to expert. 

Another significant training issue is mentoring (for a review, see 
Hoffman & Ward, 2015). Specifically, we currently have no empirically 
validated, robust method for identifying individuals who might become good 
mentors, or for developing career tracks for individuals to become expert 


domain mentors as well as domain experts. 


What Is the Role of Affect in Expert Decision Making? 


When discussing the role of affect in expert decision making, it is important 
to distinguish between integral, task-relevant affect and incidental emotions 
that are extraneous to the problem at hand. Experts have been shown to be 
selectively sensitive to task critical information (cf. Chi et al., 1988) and, 
unlike laypersons, may differentially respond to task-related and unrelated 
affect. This means that experts’ metacognitive control enables them to 
identify potentially intrusive emotional responses that are not relevant to the 
task at hand and control them. Expert pilots, for instance, report that they are 
cognizant of emotions that might negatively influence their decision making, 
such as aggravation or agitation (e.g. Mosier et al., 2009). A notable 
exception is the “startle effect” generated by a sudden, surprising, and 
stressful event, as was the case in the Air France 447 loss of control over the 
Atlantic (BEA, 2012). Researchers have suggested that practice focused on 
attentional behavior and sense-making during startle situations may be an 
additional competence area in the development of expertise (Casner, Geven, 
& Williams, 2013; Hilscher & Kochan, 2005). 


On the other hand, experts are well attuned to affect that is in response 


to critical elements of the task context and that may have significance for 
their decisions. For experts, the affective reaction to a situation — particularly 
comfort or discomfort — may represent a knowledge-based informational cue, 
and may be a salient component of their decision making. Emotional cues 
may have signal character that aids recognition and triggers appropriate 


responses without much deliberation. Dominguez (2001), for instance, 


reported that physicians frequently refer to their comfort level while deciding 


on whether or not to continue with laparoscopic surgery. This function of 
affect is similar to the role of “hunches” in split-second decision making. 

Experts’ emotional reactions to cues may also provide a frame for their 
sense-making and guide their information search and integration. For 
example, Fischer, Orasanu, and Davison (2003) studied pilots’ risk 
assessment during a hypothetical windshear situation at takeoff, and observed 
that pilots apparently used their affective reactions to changing flight 
conditions — for example their level of discomfort — as relevant information 
to frame their decision making. 

When a situation is not recognized as familiar, affective responses such 
as unease or discomfort can motivate the expert to engage in more 
information gathering, or more substantive sense-making processes. Affect 
can also provide a metric for mental simulation. If the expert does not feel 
comfortable with the probable outcome of an imagined course of action, 
he/she will likely seek another option. These suggestions need to be 
substantiated by further research into the role of task-relevant affect in expert 


decision making. 


NDM Cross-Cultural Research 


NDM researchers have begun to study decision making across cultures and to 
assess whether NDM methods, in particular CTA, can be employed with 
individuals from diverse cultures (Rasmussen et al., 2013). Some work has 
looked specifically at cultural sense-making. For instance, Sieck, Smith, 
Grome, and Rababy (2011) used CDM to identify cultural knowledge that 
influenced the decisions by military personnel during crowd management in 
the Middle East, and used these insights to develop a training program for US 
military personnel. This work demonstrates that NDM is a promising 
approach to understanding cultural factors in experts’ sense-making in a 
cross-cultural setting, and to preparing professionals for work in international 


contexts. 


Extending NDM Methods 


Simulation tools and other technologies such as eye-trackers and video 
cameras offer a mechanism for research and training. For example, some 
work has investigated expert—novice differences in surgeon cognition using 


eye tracking technology (e.g. Law, Atkins, Kirkpatrick, & Lomax, 2004), and 


has examined expert decision processes of fire service commanders using 
helmet-mounted video cameras (e.g. Cohen-Hatton, Butler, & Honey, 2015). 
High-fidelity simulators have long been used in airline operations, and 
early NDM research took advantage of the availability of simulation data to 
examine decision processes in professional pilots (e.g. Mosier, 1990). 
Simulators of increasingly high fidelity and Immersive Simulated Learning 
Environments create interactive, sometimes geographically distributed 
environments that mimic the complexity and psychological dynamics of their 


real-world counterparts (Alison et al., 2013). Complex military command and 


control simulations have been used to examine the impact of technology on 


ad hoc team operations (Strater et al., 2011), and to prepare military officers 


for operational decision making (Zimmerman et al., 2011). Immersive 
simulation platforms are effective in training and measuring team 
coordination among agencies in disaster response and in identifying pitfalls 


and counterproductive practices (Alison et al., 2013). Simulations are 


increasingly being used for research and training in other domains as well 
such as healthcare (e.g. operating rooms; Stefanidis, Korndorffer, Markley, 
Sierra, & Scott, 2006) and offshore well control (Roberts et al., 2015). 


NDM methodologies hold promise for the development of computer 


tools that support the representation of complex environments across 


distributed teams and multi-team systems. Tools must enable sharing not only 
a “common picture” but also the history (and learning) behind it, its meaning, 
areas of uncertainty or ambiguity, and its implications. Tools should include 
enhanced representational systems (including concept mapping and domain 
ontologies) that support the process of thinking in teams, team 
communication, and the development of shared understanding — as well as 
process redesign for problem-solving, reasoning in teams under conditions of 
high complexity and inherent uncertainty, and anticipation of decision and 


behavioral consequences. 


Technology and NDM 


The world in which experts operate today has changed in significant ways 
since NDM was conceived in the 1980s. Technology has become more 
prevalent and sophisticated and its function has been transformed: no longer 
simply an aid or tool for human operators, in some domains it is now 
conceived of as a team member (as can be seen in research in 
human—automation teaming or human-—robot teams; Demir & McNeese, 
2015; Johnson, Bradshaw, Hoffman, Feltovich, & Woods, 2014). This 
technological shift also has important implications for NDM theory and 


research. 


Hybrid Ecologies 


The capability of electronic systems to reduce the ambiguity inherent in 
naturalistic settings has transformed many decision domains into complex, 


hybrid ecologies (Mosier, 2008). In a hybrid ecology, cues originating in the 


extemal, physical environment must be integrated with information from 
internal, electronic systems. This is an important issue for NDM because the 
two sides of the ecology may require different types of processing. On the 
naturalistic side, information and cues are absorbed primarily via sensory and 
kinesthetic mechanisms: the look of the weather, smell of smoke, feel of the 
physical environment. On the electronic side, information requires more 
analytical cognitive processing: the parameters of the system, status readouts 


of the patient, precise position of the vehicle. 


Sense-Making in the Hybrid Ecology 


To a great extent, sense-making is a matter of ensuring and maintaining 
consistency among indicators. Accomplishing this demands some analysis, as 
experts are required to think logically and analytically about data and 
information. They need to know what data are relevant, integrate all relevant 
data into a “story” of the situation, and ensure that the story is rational and 
appropriate. Macrocognitive strategies for problem detection such as 
attention management and developing mental models are critically important 
for recognizing inconsistencies in data, as well as for understanding the limits 
of technological systems and recognizing their strengths and inadequacies. 

A critical component of expertise in hybrid environments is knowing 
whether and when the situation is amenable to intuitive recognition. This 
entails the ability to recognize not only patterns of cues and information, but 
also potential pitfalls that are artifacts of technology, such as mode errors, 
hidden data, or non-coupled systems and indicators. Expertise includes 
knowledge of how a system works, and the ability to describe the functional 


relations among the variables in a system (Hammond, 2000). It entails being 


able to evaluate what a given piece of data means when shown in a particular 
color, position on the screen, configuration, and/or system mode. Clearly, 
NDM research and design in hybrid ecologies — domains dominated by 
technology and automation — must take into account the complexity of the 
human—automation systems that will be performing within them and the 


nature of the expertise required. 


Technology Design for NDM 


As technology becomes ubiquitous in NDM environments, it will be critical 
that its design facilitates expert decision making. Decision-Centered Design 


(DCD; Hutton, Miller, & Thordsen, 2003) is a systems engineering approach 


developed by NDM researchers that is based on an understanding of the 
expertise of users. Systems can interfere with expertise and reduce 
performance, for instance by presenting too much data, or presenting high- 
level understanding rather than low-level details. DCD helps to shape the 
design of information technologies to support different aspects of expertise, 
such as making fine discriminations, anticipating events, seeking diagnostic 
data, detecting early signs of problems, and seeing the big picture. The core 
of the design process is knowledge elicitation, which captures what experts 
know, think, and do in performing their tasks. DCD specifies the cognitive 
challenges involved in the task of interest, including the requirements and the 
context of the task, and elicits information from experts about these 
challenges to create systems that support cognitive performance (Stanard, 
Uehara, & Hutton, 2003). 

Militello et al. (2015) considered the design of portable and wearable 
computing devices to be used in the extreme environment of military 
pararescue. They applied CTA and knowledge audits to elicit incidents 
highlighting experts’ knowledge and skills, and then expanded on these 
incidents to identify decision support opportunities and critical cognitive 
requirements. The authors identified important caveats in the design of 
information technology for experts, including the notion that technological 


innovations must not hinder key aspects of expert performance. For example, 


when data are transformed into information, the capability to recognize 
patterns and anomalies may be eliminated, and cognitive work may be 
increased (e.g. Patterson, Militello, Su, & Sarkar, 2016b). Technology and 
system design must also address issues of uncertainty and risk management 
(Grote, 2014). Enhanced tools developed for concept mapping, such as 
CmapTools (Canfas et al., 2004), may also be adapted to aid in representing 


and communicating complex problems and solutions. 


Multi-Team Systems 


A relatively new challenge for NDM is the multi-team context where two or 
more teams need to collaborate toward the accomplishment of collective 


goals (Mathieu, Marks, & Zaccaro, 2001; see Sonesh, Lacerenza, Marlow, & 


Salas, Chapter 27, this volume). New training and infrastructure will be 
required to support remote collaboration and the integration of people, 


technology, and information (e.g. Haavik, 2011). Ultimately, NDM research 


will need frameworks to describe processes within human—human and 
human—automation teams, and to define how time pressure or social stressors 
(e.g. goal conflicts) within groups impact complex cognition and behavior. 
We have made much progress since the 1980s in defining, examining, 
and training expert decision making in naturalistic environments. “Old” 
methods such as CTA and CDM continue to provide insight into expertise 
and guidance for expert development. Recent concepts such as 
macrocognition, data/frame, and flexecution, have enabled us to describe 
expert decision processes more precisely. New technology has provided the 
means to examine and train expert decision processes more faithfully. In 
order for the NDM community to make significant contributions to future 
challenges, existing NDM methodologies and theories must take a 
multidisciplinary approach to address the “multi-ness” that characterizes the 
current world — multiple partners, multiple options, multiple dilemmas, 
multiple domains, multiple initiatives, multiple goal conflicts, multiple 


responsible actors, and multi-team systems and collaborative response. 
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Introduction 


Over the past decade several landmark studies have advanced our scientific 
understanding of decision making skill, its measurement, and its acquisition 
(e.g. Skilled Decision Theory). Here we present an integrative review of 
skilled human decision making in experts and non-experts, with emphasis on 
four emerging insights. (1) Among non-experts, normatively superior 
decision making is associated with a domain-general skill that has largely 
been neglected in research on general intelligence. (2) Statistical numeracy 
tests (i.e. assessments of practical probabilistic reasoning) tend to be the 
strongest single predictors of general decision making skill across wide- 
ranging numeric and non-numeric judgments and _ decisions 
(www.RiskLiteracy.org). (3) The superior decision making exhibited by 
experts and non-experts primarily reflects specialized knowledge and 
integrated long-term memory representations that inform adaptive heuristic 
Strategies (i.e. representative understanding rather than _ rational 
optimization). (4) High levels of basic cognitive abilities, such as fluid 
intelligence or attentional control, are not generally required for skilled or 
expert decision making. 

Although we have endeavored to minimize jargon in this chapter, some 
clarifications merit consideration. Historically, researchers have distinguished 
between judgments (e.g. estimates) and decisions (e.g. choices), based on 
traditions from the 1940s (e.g. decision researchers followed conventions in 
economics and statistics, while judgment researchers followed conventions in 


perception). Here, and for general purposes, the terms judgment and decision 


making are roughly synonymous (e.g. a decision is a judgment about what to 
do; Baron, 2008). Thus, general decision making skill refers to stable 
differences in judgment and decision making quality exhibited across diverse 
and wide-ranging domains (e.g. health, wealth, and happiness). Likewise, 
verifiable expert decision making refers to stable differences in judgment and 
decision making quality exhibited within a specific domain (e.g. surgery, 
engineering, finance, chess). Following standard conventions, we use general 
intelligence as an umbrella term referring to a broad latent intelligence 
construct derived from interrelations among constituent basic cognitive 
abilities (e.g. fluid intelligence, crystallized intelligence, attentional control, 
memory, and others). We use skill to generally refer to acquired types of 
knowledge, skills, abilities, and related capacities. Finally, bias follows 
technical conventions referring to a tendency that does not necessarily imply 
error. Thus, biases may or may not be adaptive in various contexts (e.g. a 
“look left” bias when stepping into the street is much more adaptive in the 
United States than in the United Kingdom). 

Our review includes five sections. First, we present a summary of the 
connections between statistical numeracy, general decision making skill, and 
normative decision standards. Second, we review mechanisms that give rise 
to verifiable expert decision making and skilled decision making in general 
(Skilled Decision Theory). Third, we review psychometric studies of basic 
cognitive abilities, discussing when and why numeracy tests out-predict fluid 
intelligence tests. Fourth, we discuss simple decision aids and training 
programs that causally (and often dramatically) improve risk comprehension 
and skilled decision making. Finally, we close with a brief summary 


including ethical and policy implications. 


Numeracy and Decision Making Skill 


Since the 1990s, research on the role of mathematical skills in decision 
making has grown from practical efforts to improve risk communications and 
informed decision making, particularly in health and medicine. Early work 
leveraged lessons from the National Assessment of Adult Literacy (Kutner, 
Greenberg, Jin, & Paulsen, 2006) and the Programme for International 
Student Assessment (Breakspear, 2012). A central focus was on the acquired 
skill numeracy, which refers to the “array of mathematically related 
proficiencies that are evident in adults’ lives ... including a connection to 
context, purpose, or use ... for active participation in the democratic process 
and ... in the global economy” (Ginsburg, Manly, & Schmitt, 2006). 
Accordingly, numeracy research is generally concerned with effective 
everyday problem-solving for activities like evaluating medical treatment 
options, political claims, or financial products (Cokely, Galesic, Schulz, 
Ghazal, & Garcia-Retamero, 2012; Cokely, Ghazal, Galesic, Garcia- 
Retamero, & Schulz, 2013; Cokely, Ghazal, & Garcia-Retamero, 2014; 
Newall, 2016; Reyna, Nelson, Han, & Dieckman, 2009; Steen, 1990). Based 
on theoretical and historical connections (Huff, 1954; Paulos, 1988), 
behavioral decision research on the role of numeracy began to rapidly 
advance following the introduction of a simple three-item psychometric 


numeracy test (Schwartz, Woloshin, Black, & Welch, 1997; see also Lipkus, 


Samsa, & Rimer, 2001). The seminal contributions by Schwartz and 
colleagues showed that (1) well-educated individuals often couldn’t 


accurately answer basic numeracy questions (e.g. couldn’t convert 1 in 1000 


to 0.1 percent) and (2) numeracy scores robustly predicted the accuracy of 
disease risk interpretations. As of 2016, hundreds of studies have used 
variants of these classical-type numeracy tests to predict high-stakes real- 
world decisions. 

Over the past five years, brief adaptive and item-response instruments 
have set the standard for numeracy assessment, including the most widely 
used modern tests validated for use with diverse samples from industrialized 
communities — i.e. the Berlin Numeracy Tests (Cokely et al., 2012, 2014; see 
also Garcia-Retamero & Cokely, 2017; Weller et al., 2013). Initial analyses 
based on 21 studies, including data from 15 countries, found that the Berlin 
Numeracy Tests were the strongest single predictors of individual differences 
in general decision making skill, including the ability to evaluate and 
understand risk (i.e. risk literacy) across numerical and non-numerical 
evaluations of consumer products, medical treatments, and natural hazard 
forecasts. The three-minute adaptive Berlin Numeracy Test more than 
doubled the predictive power of the best available alternative numeracy 
instruments, uniquely predicting decision quality independent of general 
cognitive abilities (e.g. cognitive reflection, working memory, fluid 
intelligence). More than 100,000 people from 166 different countries have 
now taken one of the Berlin Numeracy Tests (adult skill sensitivity ranges 
from about the 10th to the 90th percentile in developed countries; see 
www.RiskLiteracy.org for tests and a test-version recommendation tool). 
Hundreds of subsequent studies with diverse participants from at least 60 
countries — including surgeons, lawyers, patients, children, scientists, military 
veterans, police officers, athletes, older adults, nationally representative 
samples, and others — further document the unparalleled ability of statistical 


numeracy (i.e. practical probabilistic reasoning) to uniquely predict decision 


making skill and risk literacy across (i) naturalistic, high-stakes, real-world 
choices (e.g. HIV prevention, cardiovascular risk mitigation, professional 
judgment of surgeons and physicians, public policy evaluations, natural 
hazard vulnerability, and many others) and (ii) theoretically essential 
paradigmatic tasks (e.g. risky prospect evaluation, framing resistance, sunk 
cost biases, recognizing norms, overconfidence, and others; Cokely et al., 
2014; Garcia-Retamero & Cokely, 2011, 2013, 2014, 2017; Garcia- 
Retamero, Wicki, Cokely, & Hanson, 2014; Petrova, Kostopoulou, Delaney, 
Cokely, & Garcia-Retamero, 20175). 


The causal mechanisms linking numeracy and decision making skill are 


manifold, including metacognitive, heuristic, intuitive, affective, subjective, 


gist-based, and number-sense processes (Cokely & Kelley, 2009; Ghazal, 


Cokely, & Garcia-Retamero, 2014; Lindskog, Winman, Juslin, & Poom, 
2013; Peters et al., 2006; Peters & Bjalkebring, 2015; Peters, Hibbard, Slovic, 
& Dieckmann, 2007; Reyna, 2004, 2008; Schley & Peters, 2014; Thompson, 
Turner, & Pennycook, 2011; Traczyk & Fulawka, 2016). Accordingly, 


Statistical numeracy is a robust predictor of numerical and non-numerical 
decisions, including judgments about social relationships, behavioral norms, 
professional competency, and many health behaviors (e.g. ignoring a heart 
attack). In part, this broad predictive power follows because statistical 
numeracy tests are themselves representative judgment and decision making 
tasks that challenge inductive reasoning and self-regulation under conditions 
of risk and uncertainty. In other words, effective decision making in our 
complex and uncertain world often requires the same kinds of reasoning and 
metacognitive skills that are used when solving various practical probabilistic 


math problems (e.g. evaluating thoughts, feelings, and risks). To further 


clarify connections, including shared cognitive and logical elements, we next 


consider standards of rationality and normative decision making. 


Rationality and Normative Standards 


Decision science broadly involves three main projects: descriptive (e.g. what 
decisions do people make and why), normative (e.g. what decisions should be 
made and why), and prescriptive (e.g. how can actual decisions be improved 


and why) (Baron, 2008). Modern notions of rationality typically refer to 


coherence standards that may be used for the determination of normatively 
superior judgments and decisions. One of our most influential standards 
emerged in 1654 when Blaise Pascal began corresponding with Pierre de 
Fermat about the division of stakes in a popular gambling game. Although 
they were not the first to attempt to formalize chance, their letters became the 


founding documents of the logical system at the heart of modern science and 


decision theory — i.e. probability theory (Hacking, 2006). Exactly 300 years 
later a natural extension of inductive logic known as Bayesian probability 
theory enabled the axiomatization of subjective expected utility theory — i.e. 
the formal normative decision theory that is a basis of modern statistical and 
economic applications, as presented in the book The foundations of statistics 
(Savage, 1954; see also Jaynes, 2003; Schlaifer & Raiffa, 1961; Von 


Neumann & Morgenstern, 1944). To clarify, logic can be fundamentally 


divided into two major categories, namely (i) deductive logic — sound 
reasoning from premises to conclusions that are certain — and (ii) inductive 
logic — sound reasoning from premises to inferences that involve risk (e.g. 
characterized by known probability distributions) or uncertainty (e.g. 
characterized by unknown probabilities or exposure) (Holland, Holyoak, 


Nisbett, & Thagard, 1986; Savage, 1954). Because we live in a 


fundamentally risky and uncertain world, the practical decision-relevance of 
deductive logic typically pales in comparison to that of inductive logic. 

Theoretically, the goal of all rational decision making is to get more of 
what one should want (Baron, 1985, 2008; Hastie & Dawes, 2010). “Should” 
is complicated and requires many philosophical and value assumptions. 
Setting aside philosophical issues, good decisions can be defined by logical 
processes that coherently maximize desired outcomes in accord with 
integrative optimization techniques (e.g. determining the maximum or 
minimum value of a function subject to constraints as in formal cost-benefit 
or decision analysis). To illustrate, consider the example of deciding among 
several modest risky financial prospects such as lotteries involving two 
choices (e.g. (a) gain $100 for certain versus (b) 75 percent chance of gaining 
$200). Given enough choices like these and some other basic assumptions, on 
average a normatively superior decision would result when selecting options 
with the highest expected value — i.e. multiplying the probability of an 
occurrence by its value (e.g. 75% * $200 = $150, which is more than $100, 
implying that a risky choice should be favored). In more naturalistic cases, 
whether evaluating a credit card offer or the personal advice of a friend, the 
logic is functionally the same. Rational, normatively superior decisions may 
be defined by optimization analyses that coherently integrate values, goals, 
preferences, and constraints in accord with standards of logic, probability, 
and statistics (Baron, 1985, 2008; Edwards, 1954; but see Gigerenzer, Todd, 
& the ABC Research Group, 1999). 


It is noteworthy how well most people’s decisions approximate various 


normative standards, as if they actually solved an econometric or Bayesian 
Statistical equation (Chater, Tenenbaum, & Yuille, 2006). However, beyond 


shared conceptual elements, optimization methods bear little resemblance to 


actual human decision processes: most people do not compute statistical 
analyses in their head for the hundreds of decisions they make every day (e.g. 
selecting shoes or breakfast cereals, deciding who to talk with and what to 
discuss). In many naturalistic contexts decisions actually entail so much 
complexity and uncertainty that comprehensive optimization is impossible 
even for the most powerful computers. Because people have limited time, 
knowledge, and cognitive resources, alternative decision strategies are 
required, leading people to rely on the simple heuristics that so often 
empower effective decision making (e.g. satisficing, take-the-best, 
recognition, fluency; Gigerenzer et al., 1999; Klein, 1999; Simon, 1956, 
1990). Taken together, the literature reviewed here indicates (a) that 
Statistical numeracy robustly predicts general decision making skill and (b) 
that skilled decision making is fundamentally about reckoning with risk and 
uncertainty. However, (c) the link between numeracy and superior decision 
making cannot be explained by differences in the use of formal optimization 
methods because (d) most decisions are ill-structured, making them so 
complex that optimization is practically impossible. Thus, a central question 
remains: What are the causal mechanisms that give rise to normatively 
superior human decision making? To address this question, we turn to 


research on skilled decision making in experts and non-experts. 


Expert and Skilled Decision Making 


Human cognition, including analytical and emotional processing, is often 
characterized with respect to the interplay of intuition and deliberation (e.g. 
automatic versus controlled, fast versus slow thinking, hot and emotional 
versus cold and calculating, dual-system, dual-types, and many others; Evans 
& Frankish, 2009; Kahneman, 2003, 2011; Stanovich & West, 2000; Shiffrin 
& Schneider, 1977; but also see Arkes, 2016; Cokely, 2009; Moshman, 2000; 


Osman, 2004). Common assumptions suggest that superior decision making 


may generally require outstanding cognitive abilities that enable the 
inhibition of emotions and intuitions, while empowering complex and 
abstract logical reasoning. This perspective offers something like a Mr. Spock 
depiction of the pinnacle of skilled human decision making (i.e. the 
extremely logical and emotionless science-fiction superintellect). Clearly, 
some professionals do routinely perform technical or formal decision 
analyses for high-stakes decision making (e.g. operations research). 
Computerized decision support technologies are also increasingly used to 
inform diverse decisions (e.g. forecasting, medical decision making). 
However, beyond these few examples, research reveals a very different 
picture of the underlying cognitive dynamics involved in skilled decision 
making, as extensively documented in the scientific literature on human 
expert performance (for state-of-the-science reviews see Ericsson, 1991; 
Ericsson, Charness, Hoffman, & Feltovich, 2006; Ericsson, Prietula, & 
Cokely, 2007; and this volume). 


Expert Performance 


The term expert colloquially refers to trained professionals with experience or 
credentials (e.g. a qualified expert). In contrast, expert performance is a 
scientific term referring to the verifiable, reproducible, and superior human 
performance that can be exhibited (and studied) in representative and 
naturalistic tasks. The magnitude of the superiority exhibited by expert 
performers tends to be remarkable (e.g. 3 to 1000+ standard deviations better 
as compared to novices). For example, many chess grand masters can 
simultaneously play hundreds of games, readily beating nearly all their highly 
skilled and motivated competitors (e.g. the world record involved 604 
simultaneous matches with only 8 losses). Research on these kinds of 
remarkable abilities, based on the expert performance approach, has revealed 
a great deal about the mechanisms that mediate and govern verifiable 
expertise. For example, no one ever becomes an expert in an established 
domain of expert performance without first deliberately practicing for 
thousands of hours over many years (Ericsson, Krampe, & Tesch-R6mer, 


1993; Ericsson, Prietula, & Cokely, 2007). Deliberate practice specifically 


refers to specialized, high concentration training efforts, often completed in 
solitude. These efforts are intentionally designed to move the expert past their 
current performance level through elicitation of specific feedback about 
performance weaknesses and strengths. Accordingly, deliberate practice 
tends to be quite challenging and cannot be routinely sustained for great 
periods of time (e.g. perhaps four to five hours per day on average). Unlike 
playing golf for fun with friends, one might deliberately practice by 


independently working on a very specific kind of bunker shot under 


controlled conditions. In time, careful and consistent deliberate practice 
efforts tend to produce remarkable differences in knowledge, skills, and 


abilities. 


Expert Decision Making 


Studies of chess expertise efficiently illustrate the cognitive mechanisms that 
support human expert decision making more generally (for a review see 
Gobet & Charness, Chapter 31, this volume). Decision research in chess is 
facilitated because the rules are well specified and the goals are 
unambiguous. The game has a long tradition with formal scoring conventions 
that index specific ratings for a huge number of past and present players. 
Chess is also computationally complex: there is currently no known solution 
or optimal strategy that can ensure a victory regardless of an opponent’s 
move. Thus, consistently superior decision making in chess involves heuristic 
deliberation and logical inductive reasoning under risk and uncertainty, in 
accord with formal foundations of normative decision and rational choice 
theories. As such, chess computer programs attempt to approximate an 
optimal solution by searching hundreds of millions of positions per second 
using heuristics to estimate the relative expected values of candidate moves. 
In contrast, extended search and formal computation play an exceedingly 
small role among human experts. Instead of selecting moves based on 
massive iterative search processes, verifiable experts rely on their vast and 
integrated stores of specialized knowledge in long-term memory, facilitating 
rapid encoding of goal-relevant features in tandem with sophisticated and 
nuanced pattern recognition. This integrated understanding allows experts to 
quickly narrow their search and evaluation so they can deliberatively evaluate 
a small number of outstanding candidate options (e.g. typically around four 


per minute). 


The comparison of chess experts to less skilled chess players reveals 
several notable skill differences. Experts can extract useful patterns and 
relations from information faster and in parallel, while less skilled individuals 
rely on slower serial encoding and search. In naturalistic non-routine 
situations experts tend to deliberate more during move selection compared to 
less skilled individuals, and this difference tends to benefit their performance 
(Moxley, Ericsson, Charness, & Krampe, 2012). Most importantly, experts 
consider qualitatively different moves than those considered by less skilled 
individuals. This reflects substantial differences in their fundamental 
understanding and knowledge of game situations: expert decision-makers 
have access to more and more sophisticated chunks in long-term memory 
(e.g. have memorized more than 100,000 moves and sequences). Ultimately, 
experts rely on larger intricately structured knowledge databases in memory 
to support and evaluate a small number of complex inductive mental 
simulations. Taken together, these findings highlight the power and primacy 
of knowledge and sophisticated understanding over formal computation in 


verifiable expert decision making (e.g. Ericsson et al., 2006; Klein, 1999). 


General Decision Making Skill 


Given that even experts don’t make decisions by trying to imitate formal 
mathematical optimization methods, perhaps it is not surprising that skilled, 
non-expert decision-makers also typically forgo formal calculation across 
wide-ranging everyday decision contexts. This pattern is well illustrated in 
the first cognitive process tracing study to directly map the links between 
decision strategies, cognitive abilities, and superior decision making under 


risk (Cokely & Kelley, 2009). Using choice outcome modeling, together with 


decision reaction time and retrospective verbal protocol analysis (Ericsson & 
Simon, 1980, 1993; Fox, Ericsson, & Best, 2011), the study mapped the 


Strategies that individuals with higher cognitive ability scores (i.e. working 


memory, numeracy, and cognitive reflection) used to make superior decisions 
when tasked with paradigmatic risky prospect evaluation. Analyses revealed 
that only a very small proportion of people attempted to explicitly calculate 
expected values (i.e. < 5 percent explicitly multiplied probabilities by values). 
Instead, the vast majority of superior decision making was found to be linked 
to personally relevant, affectively charged heuristic-based evaluation. 
Structural modeling further revealed that all general ability-to-performance 
relations were fully mediated by large differences in affective and elaborative 
long-term memory encoding and metacognitive reasoning (i.e. evaluating 
relations between feelings, thoughts, and consequences in personally relevant 
concrete mental simulations and narratives). 

Results from Cokely and Kelley (2009) suggested that even during ultra- 
simplified paradigmatic risky prospect evaluation, general decision making 


skill tends to reflect differences in how and how much people think about, 


and meaningfully understand, a decision problem (see also Ghazal et al., 
2014; Jasper, Bhattacharya, & Corser, 2017; Pachur & Galesic, 2013; Peters, 
2012, 2017; Reyna et al., 2009). Rather than calculating expected outcomes, 


most skilled decision-makers spent more time imagining how changes in 
wealth would affect their lives and how those changes might feel via informal 
narratives (e.g. “well that’s probably never going to happen but if it did how 
could I forgive myself”). Indeed, some of the least cognitively “able” 
individuals were among the most skilled decision-makers, reflecting their 
more extensive, personally meaningful deliberation. This sort of active, 
careful, and open-minded exploration accords with theories of successful 
decision making as detailed in Baron’s touchstone work Rationality and 
intelligence (1985). Today many related studies have been conducted in our 
labs and others, including one study with more than 50,000 participants from 
46 different countries (Rubinstein, 2013). Additional converging process 
tracing evidence comes from protocol analyses, eye-tracking, reaction time 
analyses, choice modeling, memory analyses, and causal experimentation 
(Garcia-Retamero & Cokely, 2013, 2014, 2017; Garcia-Retamero, Cokely, 
Wicki, & Joeris, 2016b; Ghazal et al., 2014; Okan, Garcia-Retamero, Cokely, 
& Maldonado, 2015; Woller-Carter, Okan, Cokely, & Garcia-Retamero, 
2012). Taken together, results suggest that general decision making skill 


often involves and can be predicted by both quantitative and qualitative 
differences in heuristic-based deliberation and representative understanding 
in long-term memory. These findings contribute to an integrative theoretical 


account of skilled human decision making. 


Skilled Decision Theory 


Skilled Decision Theory explains the essential causal mechanisms that enable 
skilled decision making in experts and non-experts with reference to the 
pivotal roles of heuristic deliberation and representative understanding. 
Skilled Decision Theory is grounded in Skilled Memory Theory (Ericsson, 


Chase, & Faloon, 1980), extending our understanding of expert decision 


making in accord with frameworks for rational thinking (Baron, 1985, 2008) 
and adaptive heuristic decision making (Gigerenzer et al., 1999). The causal 
mechanisms that support general decision making skill are similar to those 
that support complex situation model development in skilled reading 
comprehension (i.e. an acquired domain-general skill) and high-fidelity 
situation awareness in expert decision making (i.e. an acquired domain- 
specific expertise). Because most adults have a vast expert-like knowledge of 
themselves (e.g. experiences, values, habits, goals, desires, and preferences) 
personally meaningful heuristic deliberation facilitates rapid detection and 
detailed encoding of relevant information in durable, integrated long-term 
memory representations (e.g. mental models, situation narratives, and 
inductive simulations; Holland et al., 1986). By utilizing extensive and 
thoroughly integrated pre-existing knowledge structures, non-expert decision- 
makers functionally circumvent attentional capacity limitations of short-term 
(working) memory that could otherwise constrain the complexity and 
precision of their understanding and reasoning, thereby leveraging the same 
kinds of long-term working memory capacities that support expert 


performance more generally (Ericsson & Kinstch, 1995). 


Despite substantial benefits, personally oriented heuristic deliberation 
can increase the likelihood of potentially counterproductive processes (e.g. 
confirmation bias, anchoring, attribute substitution, biased sampling, small 
sample overgeneralization, affective overshadowing, over-weighting priors, 
and others). In part, this is why statistical numeracy and associated 
metacognitive skills are essential components of general decision making 
skill. Decision-makers who are skilled in practical inductive reasoning (e.g. 
Statistically numerate and metacognitively savvy), and who also engage in 
elaborative heuristic deliberation, are well prepared to correct or circumvent 
potentially costly mistakes via metacognitive heuristics (e.g. disconfirming, 
reframing, resampling, double checking, base rate conditioning, affect 
recalibrating, and coherence checking; Cokely et al., 2012; Ghazal et al., 
2014; see also Baron, 1985, 2008). Thus, ordinary people are able to generate 
a detailed, relatively coherent, and representative understanding of the 
decision problem allowing them to intuitively yet precisely feel the weight 
and potential consequences of various options and outcomes (Peters, 2012; 
Petrova, Garcia-Retamero, & Cokely, 2015; Petrova, van der Pligt, & 
Garcia-Retamero, 2014; Petrova et al., 2017a; Traczyk & Fulawka, 2016; but 


also see Fuzzy-Trace Theory for a perspective emphasizing benefits of 
imprecise, automatic aspects of comprehension; Reyna, 2004, 2008; Reyna & 
Brainerd, 1995). In turn, this understanding informs the selection of adaptive 
heuristic strategies such that decisions can approximate normative standards 
without any formal optimization analyses (e.g. ecologically rational 
representation of cue validities, cue orderings, and other factors that calibrate 


fast and frugal heuristic use; Gigerenzer, 2015; Gigerenzer & Goldstein, 


1996). In these and other ways, essential skills promote resilient and adaptive 


decision making under conditions of complex risk and uncertainty, without 


requiring or guaranteeing neo-classical rationality. 

In summary, among typical adults, including non-experts and experts 
alike, Skilled Decision Theory holds that superior decision making is 
primarily driven by (i) skilled and personally relevant heuristic deliberation 
and (ii) sophisticated, affectively charged representative understanding that 
(iii) interacts with and informs adaptive heuristic use (Cokely et al., 2012; 
Cokely, Schooler, & Gigerenzer, 2009; Garcia-Retamero & Cokely, 2017; 
Gigerenzer & Gaissmaier, 2011; Keller, Cokely, Katsikopoulos, & 
Weewarth, 2010). Taken altogether, the observed primary roles of acquired 
knowledge and heuristic deliberation sharply contrast with long-standing 
assumptions about the importance of abstract deductive logical reasoning 
capacities. To help resolve this apparent inconsistency, we next review recent 


and major psychometric studies of general cognitive abilities. 


General Intelligence and Decision Making 
Skill 


In 1994 a controversial book was published entitled The bell curve: 
Intelligence and class structure in American life (Herrmmstein & Murray, 
1994). This book was presented as a crowning synthesis of more than a 
century of research and theory on individual differences in intelligence. The 
book also included a new major study of links between general intelligence 
and life outcomes, suggesting wide-ranging potential implications, broadly 
arguing that: (a) general intelligence was one of the most influential causes of 
economic and social prosperity in the United States (and probably elsewhere) 
and (b) intelligence was largely but not entirely determined by genetic factors 
(e.g. roughly stable with firm upper-bounding individual limits). Ultimately, 
the book detailed a common perspective suggesting that the primary reason 
intelligence predicts life outcomes is because intelligence tends to be a strong 
determinant of decision making quality. According to this view, only a small 
number of people have the rare intellectual aptitudes that are required for 
consistently good decision making (“cognitive elite”), given the increasing 
social and technological complexities of our industrialized world. Indeed, a 
huge and orderly body of data documents a robust, albeit often modest, link 
between general intelligence and life outcomes including educational 
attainment, wealth, occupational achievement, professional advancement, 
health, and others. However, other claims remain largely unsubstantiated, 
including (a) the causal assumptions about the underlying mechanisms that 


allow general intelligence to predict life outcomes (e.g. fixed intellectual 


capacities), (b) the extrapolations about perceived intellectual trends (e.g. the 
emergence and dominance of a ruling class of cognitive elite), and (c) the 
appropriate policy prescriptions (e.g. social, educational, and economic 
segregation). 

Although links between intelligence and outcomes have been well 
established for nearly a century, until recently direct evidence on the 
connections between basic cognitive abilities, decision making, and life 
outcomes has been extremely limited. Consider what is regarded as the most 
comprehensive integrative analysis of the structure of general cognitive 
abilities and intelligence to date, namely the book entitled Human cognitive 
abilities: A survey of factor-analytic studies by John Carroll (1993). This 
monumental monograph presents a reanalysis of 460 factor-analytic cognitive 
ability studies collected over a 60-year period. Across all included datasets 
and tasks decision making was rarely mentioned, except in the context of 
very simple perceptual reaction-speed tasks. However, tests of reasoning 
abilities received extensive attention and featured prominently in Carroll’s 
estimates of the fluid intelligence factor, which was determined to be the 


strongest single factor explaining overall general intelligence (Figure 26.1). 
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Figure 26.1 The structure of cognitive abilities as identified in Carroll’s 
(1993) integrative analysis. The g represents overall general mental ability, 
while Gf, Gc, and Gy represent the main factors from the second stratum, 
excluding less influential perceptual and speed-related cognitive ability 


factors. 


Based on analyses of 176 large datasets including some 236 different 
reasoning factors Carroll (1993) suggested that three essential reasoning 
factors best explained fluid intelligence, namely (1) sequential reasoning, (2) 
inductive factors, and (3) quantitative reasoning. Careful inspection reveals 
that paradigmatic decision making skills were nevertheless broadly neglected. 
In contrast to common assumptions, the category labeled “inductive factors” 
does not broadly represent inductive logic. The half-dozen or so inductive 
tasks instead primarily require narrow types of inferences used during 
abstract deductive reasoning. Specifically, within deductive mathematical 
reasoning there is a class of “inductive inference” that is relatively 
independent of risk or uncertainty. In psychological research these kinds of 
deductive reasoning tasks — including progressive matrices, analogical 


reasoning, and categorization-type problem-solving — are more specifically 


referred to as “rule induction” tasks (Goldman & Pellegrino, 1984; Holyoak 


& Morrison, 2005). Across all these types of tasks, if a person can figure out 


the rule that precisely dictates the relations among various elements, they can 
always deduce the final answer with absolute certainty. 

To be clear, there is surely some psychological uncertainty involved in 
the completion of deductive rule-induction tasks (e.g. matrix reasoning), such 
as deciding when you should be confident you have determined the correct 


rule before deducing a final answer. However, these tasks are otherwise not 


indices of the full range of inductive logic that is characteristic of moder 
paradigmatic or naturalistic decision making. Moreover, the “quantitative 
reasoning” tasks used to measure fluid intelligence also typically failed to 
assess the full range of relevant skills, focusing primarily on conventional 
mathematical components that “require an appreciation of the quantitative 
concepts and relations, particularly as treated in mathematics in its various 
branches, from simple arithmetic to algebra, geometry, and calculus” 
(Carroll, 1993, p. 213). As a result of this narrow task-sampling, most major 
factor-analytic intelligence studies conducted in the twentieth century fail to 
incorporate a wide range of logical inductive, probabilistic, or statistical 
reasoning skills in their analyses. 

Historical coincidence partially explains why decision science 
perspectives have been neglected in psychometric intelligence research. 
Behavioral decision science is a relatively young field that began to have a 
mainstream presence in psychology and economics about 25 years after the 
cognitive revolution of the 1950s. Statistical theory and training is also a 
relatively modern invention as the axiomatic (objective) formalization of 
probability theory was not published until 1933, about two years before 
Ronald Fisher inspired a scientific revolution with The design of experiments. 
And still today highly trained and well-respected scientists often 
misunderstand and misapply statistical theory (Lindsay, 2015). Moreover, 
even conventional quantitative skills have historically been treated 
inconsistently in the intelligence literature. For example, Herrnstein and 
Murray (1994) excluded a brief (five minute) timed test of mathematical 
operations from their final analysis because it was not sufficiently correlated 
with their intelligence estimate (i.e. the data were available but excluded due 


to low factor loadings). Reanalyses of their data found that the excluded 


timed math operations test actually predicted wages as well as the part of the 
Armed Forces Qualifications Test data that Herrnstein and Murray used 
(Heckman, 1995). If indeed this very brief test of math ability was relatively 
independent of “general intelligence,” yet still predicted wages as well as the 
unrelated estimates of general intelligence, it follows that numerical abilities 
must explain important life outcomes that are otherwise missed by more 
standard general intelligence metrics. To evaluate the relative influence of 
intelligence and other cognitive abilities on general decision making skill, we 


next consider several recent landmark psychometric decision science studies. 


Psychometric Studies of Decision Making Skill 


At the turn of this century, Stanovich and West (2000) published a highly 
influential paper entitled “Advancing the rationality debate.” Their work 
reviewed individual differences in decision making, examining the link 
between general cognitive abilities and susceptibility to common heuristics 


and biases in the Kahneman and Tversky tradition (Stanovich, 1999). Based 


on analyses of correlations, Stanovich and West showed that intelligence 
predicted normatively superior decision making in young adult participants, 
suggesting (1) a positive manifold among many judgment and decision 
making abilities, and (2) a robust correlation with general intelligence as 
measured by standardized college achievement or matrix reasoning tests (i.e. 
deductive rule-induction tasks). Although their theoretical account of these 
findings has changed in fundamental ways over the last 15 years (Stanovich, 
2016; Stanovich, West, & Toplak, 2016), the groundbreaking work of 


Stanovich, West, and others paved the way for more comprehensive 


psychometric decision science investigations (see also the seminal 


contributions of Frederick, 2005; Peters et al., 2006). 


Decision Competency Assessment 


Historically, research has emphasized four core decision skill components 
known to shape decision consistency and accuracy, namely (1) assessing 
beliefs, (2) assessing values, (3) integrating beliefs and values, and (4) having 
a metacognitive understanding of one’s abilities, resources, and constraints 
(Edwards, 1954; Raiffa, 1968; Yates, 1990). Leveraging this reasoning, in 


2005 the first in a series of two landmark studies presented what was then the 


most comprehensive assessment of “plausible real-world correlates of good 
decision making ... [that broadly] span the domain of decision making skills” 
(Parker & Fischhoff, 2005, p. 1). Specifically, the study assessed and 
modeled general decision performance and ability structure across seven 
paradigmatic decision domains, namely, (1) risk perception consistency, (2) 
social norm recognition, (3) resistance to sunk costs, (4) resistance to 
framing, (5) applying decision _ rules, (6), choice path 
independence/consistency, and (7) confidence calibration (e.g. over-/under- 
confidence). Integrating the patchwork of individual difference assessments 
used in the past, Parker and Fischoff created a broad Young Decision Making 
Competency assessment (Y-DMC) and administered it to 110 diverse young 
adults. 

Parker and Fischhoff’s (2005) study revealed an anticipated positive 
manifold among paradigmatic decision tasks, which was well explained by a 
one-factor model. Analyses showed that the Y-DMC was related to other 
cognitive ability tests (e.g. vocabulary and executive control). Importantly, 
each of the seven facets also predicted various behavioral outcomes, decision 
styles, or risk factors ranging from sustaining social support and positive peer 
interaction, to delinquency, drug use, and other vulnerabilities. These 
relations remained significant and largely unchanged after statistically 
controlling for general cognitive abilities. Results further indicated that 
decision making skill largely or entirely mediated the link between general 
cognitive abilities and important life outcomes such as behavioral coping, 
externalizing behaviors, and overall at-risk status. 

Two years after the initial validation, a second landmark study in this 
series was published by Bruine de Bruin, Parker, and Fischhoff (2007). The 


research included an updated, advanced competence assessment designed for 


use with diverse adults from industrialized countries (i.e. the A-DMC; Table 
26.1). The extensive decision making assessment was completed at home by 
360 participants. Once again, a one-factor model provided an efficient fit, 
although a two-factor model improved model fit (e.g. from 30 percent to 46 
percent explained variance). Predictive validity was demonstrated using a 
Decision Outcomes Inventory (DOI), a robust self-report general weighted 
index of maladaptive real-world decision outcomes (e.g. being fined, 
overspending). General intelligence was also directly assessed using the 


Raven’s advanced progressive matrix reasoning test (fluid intelligence) and 


the Nelson-Denny reading test of verbal ability (crystallized intelligence). 


Table 26.1 Components of the Adult Decision Making Competence (A- 
DMC) assessment (Bruine de Bruin et al., 2007). 


A-DMC 
component Description Example item 
Consistency This task asks What is the probability that 
in Risk participants to judge someone will steal something 
Perception the probability of from you during the next year/in 
various events the next 5 years? 
happening in two 
different time frames. 
Recognizing ‘This test measures how First set: “It is sometimes OK to 


Social Norms 


well participants judge 
social norms. 
Participants assess 16 
undesirable behaviors. 


steal under certain 
circumstances.” 


Second set: “Out of 100 people 
your age, how many would say 
it is sometimes OK to steal 


Resistance to 
Sunk Costs 


Resistance to 
Framing 


Applying 
Decision 
Rules 


This test measures the 
ability to ignore prior 
financial and time 
investments that are no 
longer paying off when 
making decisions. 


This task measures 
whether value 
judgments are affected 
by irrelevant variations 
in how the problem is 
presented. 


This task evaluates the 
ability to apply 
decision rules, by 
asking participants to 
choose between DVD 


under certain circumstances?” 


After a large meal at a 
restaurant, you order a big 
dessert with chocolate and ice 
cream. After a few bites you 
find you are full and you would 
rather not eat any more of it. 
Would you be more likely to eat 
more or to stop eating it? 


Recent evidence has shown that 
a pesticide is threatening the 
lives of 1,200 animals. Two 
response options have been 
suggested. Which option do you 
recommend: 


(1) Option A: 600 animals 
will be lost for sure. 


(2) Option B: 75% chance 
400 animals will be lost, 
and 25% chance that 1,200 
animals will be lost. 


The same item is then presented 
in a “gain” format (e.g. 600 
animals are saved for sure). 


Lisa wants the DVD player with 
the highest average rating across 
features. Which one of the 
presented DVD players would 
Lisa prefer? 


Path 
Independence 


Under-/over- 
confidence 


answer. 


players with different 
ratings and features. 


This test presents item 
pairs posing 
normatively equivalent 
choices between 
gambles. The 
participants’ choice 
should not be affected 
by normatively 
irrelevant changes. 


This test measures how 
well participants can 
assess their own 
knowledge. 
Participants first 
answer a true/false 
question, then assess 
their confidence in that 


Which do you like best: 


(1) Flip a coin. If heads, 
win $100. If tails, win $0. 


(2) Sure Win. Win $50 for 
sure. 


Also presented in different 
forms. Performance measured 
by participant’s consistency in 
choices. 


True or false: Stress makes it 
easier to form bad habits. 


How confident are you? 


50% (just guessing) to 100% 
(absolutely sure). 


The results of Bruine de Bruin et al. (2007) revealed strong relations 
between intelligence and general decision making skill (i.e. correlations 
around 0.6). However, predictive validity modeling showed the composite A- 
DMC decision quality score predicted decision outcomes about four times 
better than intelligence scores alone. Decision making skill as assessed by A- 
DMC also explained about 75 percent of the link between intelligence and 


decision outcomes, as measured by the DOI. This finding suggests that a 


primary reason intelligence predicts better decision outcomes is because more 
intelligent people tend to acquire higher levels of decision making skill. 
Nevertheless, estimated decision making skill scores predicted decision 
outcomes much better than, and largely independently of, intelligence (i.e. 80 
percent of decision making skill’s predictive power was independent of 
intelligence). 

Taken together, these landmark studies suggest that while standard 
intelligence tests predict real-world decision making outcomes, they tend to 
be much less powerful compared to more direct measures of general decision 
making skill. Results further confirm that paradigmatic decision making 
competency provides a robust estimate of real-world decision making skill, 
predicting high-stakes decision outcomes far better than, and largely 
independent of, general intelligence (Bruine de Bruin et al., 2007; Del 
Missier, Mantyla, & Bruin, 2012; Parker & Fischhoff, 2005). 


Intelligence, Decisions, and Numeracy Components 


The work of Parker and Fischhoff (2005) and Bruine de Bruin et al. (2007) 
provided a robust and representative benchmark assessment of paradigmatic 
decision making skills including (but not limited to) tasks in the heuristics 


and biases tradition (Kahneman, 2011; Tversky & Kahneman, 1985). 


However, until recently, broader efficient research assessments covering the 
full range of quantitative numeracy skills have not been widely available. To 
address this gap we turn to a comprehensive adult numeracy framework 


(Ginsburg et al., 2006) derived following a systematic review of 29 existing 


mathematical and numeracy frameworks and related national education 


standards. This framework indicates that the modem core collection of 


essential components of adult numeracy in industrialized societies typically 
involves: operations including computation, estimation, rates, ratios, 
proportions, and percentages; probability including collection, organization, 
and display of data, analysis and interpretation of data, chance and 
probability, and inferential reasoning; geometry including measurement units, 
trigonometric ratios, angles and lines, shapes, perimeter, area, and volume, 
length, width, height, and radius; and algebra including algebraic 
expressions, symbols, equations, and functions. Building on this and related 


Statistical literacy frameworks (Gal, 2003; Ginsburg et al., 2006; Kutner et 


al., 2006), we developed the Berlin Numeracy Components Tests (BNT-C), 
using a multiphase iterative test development process, in order to provide 


simultaneous estimates of: 
1. Full-scale adult numeracy. 


2. Adult numeracy subscales (i.e. operations, probability, algebra, 


geometry). 
3. Statistical numeracy (i.e. a composite of operations and probabilities). 
4. Conventional numeracy (i.e. a composite of algebra and geometry). 


Several two-parameter logistic models were estimated using Item 
Response Theory in order to identify a final pool of (up to) 36 items for 
efficient assessment across various samples and subsamples of diverse people 
in industrialized countries (e.g. all test/sub-test information functions peaked 
around theta of zero; Cokely, Allan, Ghazal, Feltz, & Garcia-Retamero, 
forthcoming; Ghazal, 2014). As with previous tests, different formats allowed 
items to be presented in adaptive or traditional modes (e.g. paper and pencil), 


as needed for various study constraints (e.g. brief online versus extensive in- 


person testing). Thanks in part to essential funding from the National Science 
Foundation and others, we then endeavored to conduct the most 
comprehensive and representative series of studies of general decision 
making skill in history, while simultaneously updating national and 
international norms for numeracy, risk literacy, and decision making skill 
(Cokely et al., forthcoming; but for similar extensive and ambitious ongoing 
approaches that have also revealed strong and robust influences of numeracy 
see Stanovich et al., 2016). 

In one of our first broad studies, we included hundreds of paradigmatic 
and ecologically sampled decision making tasks (see Table 26.2), including 
(a) the Adult Decision Making Competence assessment (A-DMC), (b) a 
paradigmatic prospect evaluation assessment battery (e.g. risky lotteries, 
intertemporal choice), (c) a class inclusion illusion task battery (e.g. 
denominator neglect, ratio bias), and (d) an ecological risk literacy test 
battery made up of real-world decision and evaluation tasks sampled from 
representative health, financial, natural hazard, civic, and consumer contexts 
(e.g. evaluating real advertising and political polls, interpreting relevant 
medical risks, offering actual recommendation to peers about HIV, 
relationships, finances, etc.). One hundred and twenty-six young adults 
completed all phases of the assessment including all decision making tasks, 
the newly developed Berlin Numeracy Components Test (BNT-C), seven 
other leading numeracy tests, the Raven’s advanced progressive matrices for 
assessment of fluid intelligence, the assessment of cognitive impulsivity 


developed by Frederick (2005), as well as about two dozen other personality, 


trait, style, ability, demographic, and outcome assessments. 


Table 26.2 General decision making skill assessment battery. 


Component 


A-DMC 


Prospect Evaluation 


Risky prospects 
and lotteries 


Intertemporal 
choices 


Reference Class and 
Class-Inclusion 
Neglect, with 
confidence 
calibration 


Description 


Includes a battery of 
seven categories of 
paradigmatic decision 
tasks and common 
biases. 


Includes paradigmatic, 
modest-stakes risky 
prospect evaluations 
presented in many 
formats, across a wide 
range of risk and EV 
ratio ranges. 


Presented a series of 
paradigmatic time- 
reward preference tasks 
based on previous 
research. Items present 
differing time intervals 
and reward amounts. 


Measures the propensity 
to neglect reference 
classes, base rates, and 
various ratio-relevant 
decision factors (e.g. 
denominators). 


Example item 


See Table 26.1. 


Which of the two 
options do you prefer: 


(1) Lose $50. 


(2) 50% chance to 
lose $400. 


Which of the two 
options do you prefer: 


(1) $3400 this 
month. 


(2) $3800 next 
month. 


A new drug is 
introduced to reduce the 
risk of death from a 
heart attack for people 
with high cholesterol. A 
study of 900 people 
with high cholesterol 


showed that 80 of the 
800 people who have 
not taken the drug died 
after a heart attack, 
compared with 16 of the 
100 people who did take 
the drug. How 
beneficial was the drug? 
(plus confidence) 


Ecological Risk Assesses informed Imagine you take out a 
Literacy and decision making inrisky $50,000 federal student 
Informed Decision health, financial, and loan to help pay for 
Making, with natural hazard decisions college. You are offered 
confidence sampled from, and four possible repayment 
calibration ecologically plans. Given 

representative of, information about each 

common naturalistic of these plans, 

decision tasks. participants must 


determine which plan is 
best, based on different 
criteria. (plus 
confidence) 


Upon completion of the study we derived an overall general decision 
making skill estimate based on a weighted cumulative index of superior 
decision making performance (e.g. weighted by relative factor loadings). As 
measured with the Berlin Numeracy Components Test alone, full scale 
numeracy accounted for 34 percent of the total variance in overall decision 
making skill. In comparison, the best combination of all other cognitive 


ability and numeracy instruments accounted for roughly 30 percent of overall 


decision making skill, as depicted in Figure 26.2. Despite taking more than 
ten times longer to complete, all other cognitive ability tests combined 
provided significantly less predictive power compared to the single brief 
Berlin Numeracy Components Test. Analyses further indicated that the 
Statistical numeracy sub-test portion alone explained 33 percent of the total 
decision making skill variance (as a single predictor), such that 97 percent of 
the predictive power of full-scale numeracy was shared with the statistical 
numeracy subscale. This finding highlights the widely observed, robust link 
between general decision making skill and statistical numeracy (e.g. a five- 
minute test specifically focusing on operations and probabilistic inductive 
reasoning skills). By nearly any behavioral science standard this association 
is very strong. It is similar to the relationship between temperature and 
distance from the equator in the United States (e.g. Michigan is almost 
always much colder than Oklahoma or Florida), or more than 25 times 
greater than the meta-analytically derived average effect of ibuprofen for 
acute pain relief. In comparison, the predictive power of conventional 
numeracy alone (i.e. geometry and algebra) explained less than half as much 
variance as Statistical numeracy (i.e. 16 percent). To illustrate, Figure 26.2 
presents data from a sample of young adults displaying the total variance 
explained in overall superior decision making performance (i.e. general 
decision making skill) by each of the following single predictors alone: Full- 
Scale Numeracy (i.e. statistical numeracy + conventional numeracy), 
Statistical Numeracy (i.e. probability + operations), all other assessed 
General Cognitive Abilities (e.g. fluid intelligence, cognitive reflection, 
health literacy, other numeracy tests), Conventional Numeracy (i.e. algebra + 
geometry), and Raven’s Advanced Progressive Matrices (i.e. fluid 


intelligence) (Cokely et al., forthcoming; Ghazal, 2014). 


Adjusted R? 


Full Scale Statistical All Other Conventional Raven’s 
Numeracy Numeracy Abilities Numeracy Advanced 
Combined (Geometry and Progressive 

Algebra) Matrices 


Figure 26.2 The total variance explained in overall general decision 


making skill by each variable modeled as a single, sole predictor. 


Additional analyses confirmed that numeracy was strongly related to 
fluid intelligence in our study (i.e. about 25 percent shared variance), with 
fluid intelligence exhibiting a significantly stronger association with 
conventional numeracy than statistical numeracy. Fluid intelligence alone (as 
assessed by the Raven’s advanced progressive matrices) explained less than 
half the variance of the brief statistical numeracy test, as is depicted in 
Figures 26.3—26.5. Structural models further indicated that the link between 
fluid intelligence and general decision making skill was largely mediated by 
Statistical numeracy. Consistent with previous studies, fluid intelligence 
predicted general decision making skill in large part because more intelligent 
people had higher levels of statistical numeracy proficiency. That said, high 
levels of statistical numeracy and decision making skill also occurred in the 
absence of high fluid intelligence scores. Similar patterns of results were 
found in fine-grained analyses of specific decision competencies (e.g. A- 


DMC, Ecological Risk Literacy, Class-Inclusion Illusions, and Risky 


Prospect Evaluation). In all cases the strongest single predictor of decision 
making skill was full-scale numeracy, followed closely by statistical 
numeracy, which almost entirely mediated any connection between fluid 
intelligence and decision making skill. In turn, subsequent structural 
modeling revealed that one’s ability to evaluate and understand risk (i.e. risk 
literacy) tended to mediate the observed relations between statistical 
numeracy and general decision making performance: About 70 percent of the 
relationship between numeracy and decision making was explained by 
decision making performance on naturalistic risky decision making tasks (e.g. 
assessing real financial products, evaluating treatment advice, interpreting 
risks and trade-offs based on actual scientific evidence or government- 
approved brochures, etc.). These results converge, indicating that acquired 
Statistical numeracy skills robustly and uniquely predict one’s general 
decision making skill, often operating independently of fluid intelligence and 
other basic cognitive abilities among diverse and generally healthy young 
adults. To further illustrate these relations, based on a sample of young 
adults, Figure 26.3 depicts the proportion of variance in a factor-analytically 
derived best-fitting model of overall general decision making skill as a 
function of (a) Statistical Numeracy, (b) Paradigmatic Decision Making 
Tasks (e.g. risky prospect evaluation), and (c) Fluid Intelligence (i.e. Raven’s 


Advanced Progressive Matrices). 


Statistical 
Numeracy 
85% 


Paradigmatic 
Decision 
Making 
10% 
Intelligence 
5% 


Figure 26.3 The proportion of variance in a factor-analytically derived 
best-fitting model of overall general decision making skill. 
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Decision Making Skill 


Fluid 
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Figure 26.4 Spatial depiction of the estimated and approximate scaled 
relations between (a) statistical numeracy, (b) conventional numeracy, (c) 
risk literacy and general decision making skill, and (d) fluid intelligence. 
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Figure 26.5 The predicted hypothetical restructuring of Carroll’s (1993) 
cognitive ability model based on emerging data, highlighting the robust 


influence of general decision making skill. 


How Numeracy Out-Predicts Fluid Intelligence 


The tight connections linking risk, uncertainty, and decision making help 
explain why statistical numeracy is such a strong predictor of decision 
making skill: decision making is fundamentally about reckoning with risk 
and uncertainty (Figure 26.4). Accordingly, a normatively superior decision 
is essentially a good bet. If you know the rules of probability theory and 
understand your own information processing biases, competencies, 
knowledge, and values, no matter how carefully you check your reasoning, 
your choice will always entail some risk and uncertainty. In contrast, fluid 
intelligence tests are primarily about careful and thorough deductive 
reasoning under conditions of certainty. Thus, if you carefully concentrate for 
long enough to triple check the coherence of your analysis, you can 
determine the right answers on intelligence tests with perfect certainty every 
time. 

The fact that abstract deductive reasoning (i.e. rule-induction) is a 
central feature of “gold standard” fluid intelligence tests helps explain why 
these tests are argued to be best characterized as assessments of working 
memory and attentional control (e.g. general fluid memory; Kyllonen & 
Christal, 1990; McCabe, Roediger, McDaniel, Balota, & Hambrick, 2010). 


One’s ability to coherently sustain and direct high-concentration attention in 


the service of error-free analysis is especially relevant for time-limited 
problem-solving tasks involving complicated abstract rules and unfamiliar 
artificial objects. For example, a worry-free person who wakes up fresh in the 
morning might answer nearly every matrix question correctly, yet that same 


person may cut comers and make strange processing and logical errors if 


sleep deprived, hungry, upset, or otherwise preoccupied (e.g. lapses of 
attention). Indeed, many kinds of acquired, environmental, motivational, and 
emotional mechanisms influence these attentional control capacities, as 
reflected in the very large historical increases in fluid intelligence scores 
observed during the twentieth century (Fox & Mitchum, 2013, 2014; but for 
training and motivation studies see Cokely, Kelley, & Gilchrist, 2006; 
Duckworth & Seligman, 2005; Jaeggi, Buschkuehl, Jonides, & Shah, 2011). 


Because fluid intelligence tests are fundamentally tests of abstract 


(unfamiliar) reasoning under certainty, the task demands imposed by fluid 
intelligence tests are not broadly representative of typical human decision 
making (i.e. not ecologically representative or valid). In contrast, actual 
human decision making is largely a knowledge-centric and comprehension- 
oriented activity. Effective decision making generally involves consideration 
of options in the context of extensive knowledge, deeply held values, actual 
responsibilities, and practical constraints (e.g. with reference to and relevance 
for family, finances, health, career, happiness, trust, regret, safety, etc.). The 
precise evaluation of costs, benefits, and trade-offs necessarily entails 
concrete consideration of familiar, tangible, and affectively charged outcomes 
and implications (e.g. losing money, avoiding illness, giving risky advice). In 
some sense, trying to measure decision making skill with an unfamiliar 
abstract rule-induction task may be like trying to measure reading 
comprehension with an unfamiliar foreign vocabulary learning test. Even if 
the strange vocabulary learning test had some predictive power, it would be a 
poor assessment when compared to more comprehensive analyses of the full 
range of underlying cognitive processes (e.g. reading complex and relevant 
passages in one’s native language). Ultimately, statistical numeracy tests are 


robust predictors because their imposed task demands are more representative 


of the diverse sub-skills and processes typically involved in effective 
naturalistic decision making (e.g. practical and personally meaningful 
inductive reasoning and self-regulation under risk and uncertainty). For these 
and many other reasons, in the light of emerging data it is likely that more 
comprehensive psychometric studies will necessitate major restructuring of 
unified models of general human intelligence and the nature of its constituent 
abilities, as depicted in Figure 26.5. This figure depicts a hypothetical 
restructuring of Carroll’s (1993) cognitive ability model that is predicted to 
emerge as Statistical numeracy and general decision making skill are more 
broadly and accurately represented in factor-analytic studies of general 
intelligence. As depicted, based on the extant evidence, the structure indicates 
that Carroll’s current Fluid Intelligence factor appears likely to dissolve by 
dividing its factor loadings between acquired General Decision Making Skill 
(e.g. practical logical inductive reasoning skills) and General Fluid Memory 
and Learning (e.g. executive functioning, attentional control), consistent with 
recent studies on the relations between decision making skill, fluid 
intelligence, and numeracy. The model would also better accord with 
research indicating that the tests commonly used to estimate fluid intelligence 
(e.g. matrix reasoning tasks) tend to largely be measures of variations in 
coordinated general attentional control and short-term (working) memory 
storage (McCabe et al., 2010). To further explore the roles of skills and 
abilities in representative decision making tasks, we now turn to recent 


research on risk communication, training, and decision support. 


Simple, Powerful Decision Support 


Numerate decision-makers are generally more resilient against information 
distortion and choice manipulation effects. Unfortunately, a large proportion 
of highly educated and intelligent working professionals, such as physicians, 
have relatively low levels of statistical numeracy. This weakness translates 
into misinterpretations of probability expressions, causing misunderstandings 
and potentially dangerous risk communication biases (Garcia-Retamero & 
Cokely, 2017; Garcia-Retamero et al., 2016b). For example, research 
indicates that nearly 50 percent of sampled physicians cannot correctly 
answer questions like “if person A’s risk of getting a disease is 1 percent in 
ten years, and person B’s risk is double that of A’s, what is B’s risk?” 
Nevertheless, an impressive number of studies using simple graphical 
representations of numerical expressions of probability (bar and line charts, 
and icon arrays) have been found to improve decision making quality in 
diverse professionals, patients, and publics (for reviews see Garcia-Retamero 
& Cokely, 2013, 2017). In one influential study involving a large nationally 
representative sample from the United States, results indicated that providing 
visual aids in addition to numerical information about the effectiveness of 
medical treatments increased the accuracy of less numerate people’s 
judgments from less than 20 percent to nearly 80 percent. The benefits of 
visual aids essentially equated the more and less numerate individuals, as 
long as participants had some minimal graph interpretation skills (i.e. graph 
literacy). In accord with Skilled Decision Theory, visual aids and other 


transparent decision aids will tend to causally eliminate large differences in 


skilled decision making by increasing deliberative evaluation and 
representative understanding of risks in long-term memory (Garcia- 
Retamero, Cokely, & Hoffrage, 2015). For example, in a recent study of 
some 300 practicing orthopedic surgeons, deliberation was found to partially 
mediate the relations between numeracy and clinically relevant judgment in a 
control condition (i.e. interpretation of actual anesthesia risks based on a 
recent peer-reviewed clinical trial). However, in a separate condition, 
differences in decision performance were completely eliminated when people 
were given a simple visual aid that promoted better understanding and more 
extensive deliberation among less skilled surgeons. 

Transparent decision aids are specifically designed to generate an 
accurate, robust, and representative understanding of risks and information in 
long-term memory (e.g. promote a thoroughly integrated and balanced 
understanding of relevant trade-offs, options, data, and consequences). Thus, 
visual aids promote risk literacy (i.e. the ability to evaluate and understand 
risk) in many of the same ways statistical numeracy skills do, as can be seen 
in the structural process model depicted in Figure 26.6. The evidence 
showing that transparent decision aids causally improve decision making by 
enhancing representative understanding and domain-specific risk literacy, 
such as health risk literacy and natural hazard risk literacy, comes from many 
different studies (for reviews see Garcia-Retamero & Cokely, 2013, 2014, 
2017). All studies were explicitly designed to be naturalistic and ecologically 
valid, accurately reproducing actual problems that people commonly 
encounter when they evaluate personally relevant information about health, 
money, relationships, and the like. For example, tasks have included 
investigations on the accuracy of perceptions of health and disease risk, and 


risk reductions; inferences about the predictive power of medical tests and 


treatment effectiveness; assessments of subjective confidence in choices and 
risk perceptions; evaluations and integrations of emotions, trust, and 
information accuracy; assessment and trajectories of health outcomes; 
preferences and memory for health information; and changes in attitudes, 


behavioral intentions, behaviors, and high-stakes informed decisions. 
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Figure 26.6 A generalized structural process model of skilled decision 
making, depicting typical patterns of situational, cognitive, and emotional 


interdependencies in accord with Skilled Decision Theory. 


Figure 26.6 depicts a structural process model of skilled decision 
making derived from causal experimentation and formal quantitative 
structural equation modeling of high-stakes decision making in health, public 
policy, and professional domains (e.g. medical screening, cancer treatment, 
responses to the Ebola pandemic). Numeracy and decision aids (e.g. visual 
aids) support skilled decision making both directly and indirectly through 
metacognitive and risk comprehension effects (e.g. deliberation, confidence 
monitoring, and calibration), refining representative understanding and 


affective reactions (Garcia-Retamero et al., 2015). Numeracy also exerts a 


direct effect on affective responses such that both risk comprehension and 
numeracy independently influence the precision and calibration of affective 


intuitions (e.g. the relative subjective evaluation of positive and negative 


emotional reactions). In accord with Skilled Decision Theory, the model 
indicates that high-quality skilled and informed decision making primarily 
follows from the interplay of deliberative evaluation (e.g. cognitive and 
metacognitive elaboration and exploration) and representative understanding 
(e.g. precise and accurate integrated mental models of risks that inform 
metacognitive and affective deliberation), providing essential foundations for 
adaptive heuristic decision making (e.g. a representative understanding 
allows one to truly feel what’s at stake, and why, thereby ordering and 
integrating exploration, cues, and priorities so as to calibrate fast and frugal 
decision making heuristics that rely on non-exhaustive lexicographic search; 


Gigerenzer & Gaissmaier, 2011). 


Efficient General Skill Training 


As an extension of our work on the benefits of visual aids we have recently 
developed an online tutoring program that helps people learn how to interpret 
and use various graphs like bar charts, line charts, icon arrays, and decision 
trees — i.e. a graph literacy training system (Cokely et al., forthcoming; 


Woller-Carter, 2016; Ybarra et al., 2017). In theory, by developing an 


efficient and relatively brief (less than two hours) graph literacy training 
program accessible to a wide range of individuals, we reasoned that we might 
be able to reach and empower more people. That is, we aimed to help 
improve graph literacy directly so that more people could benefit from simple 
visual aids that are highly effective for many types of risk communications, 
including those who score lower on standardized general ability and 
intelligence tests. 

Beyond the many practical benefits, a noteworthy theoretical result of 
our graph literacy training program was that training translated directly into 
large improvements in specific types of skilled decision making. For 
example, in one experiment about 100 participants completed either the graph 
literacy training program or a study skills training program. They were then 
tested on ostensibly unrelated decision making tasks that did not include any 
type of visual aid. Participants in the graph literacy training condition showed 
very large improvements in scores on tests of framing effects, sunk costs, and 
class inclusion illusions as compared to the control condition (e.g. about 1.5 
standard deviation). Structural analyses confirmed that the improvements we 
observed in skilled decision making were mediated by improvements in 


graph literacy independent of other basic cognitive abilities (i.e. after 


statistically controlling for numeracy, fluid intelligence, and other cognitive 
abilities) (Cokely et al., forthcoming; Woller-Carter, 2016). 

Theoretically, risk literacy is the central necessary and potentially 
sufficient condition for skilled and informed decision making among healthy 
and motivated adults in naturalistic general decision settings. The fact that 
graph literacy training substantially improved aspects of general decision 
making skill may seem surprising (Simons et al., 2016). However, we 
anticipated these effects based on the findings showing that visual aids tend 
to be particularly beneficial for these types of tasks (e.g. bar charts de-bias 
framing effects, icon arrays de-bias ratio biases, decision trees de-bias sunk 
cost effects). In some real sense when someone learns how to use graphs to 
represent data they are learning how to build useful mental models of risks 
and relations in their long-term memory. Beyond the benefits of graph 
literacy training programs, there are many effective programs designed to 
directly develop essential probabilistic reasoning skills (Paas, 1992; Rittle- 
Johnson & Koedinger, 2001). Although it can be difficult, there is no doubt 


that skilled decision making can be dramatically improved with the right 
kinds of guidance, motivation, and deliberate practice (Arkes, 1991; Baron & 
Brown, 2012; Brase, 2014; Chang, Chen, Mellers, & Tetlock, 2016; Clegg et 
al., 2015; Eskreis-Winkler et al., 2016; Fong, Krantz, & Nisbett, 1986; 
Garcia-Retamero, Cokely, Ghazal & Joeris, 2016a; Larrick, 2014; Larrick, 
Morgan, & Nisbett, 1990; Mellers et al. 2014, 2015a, 2015b; Morewedge et 
al., 2015; Peters, 2017; Peters et al., 2017; Soll, Milkman, & Payne, 2015; 
Torgerson, Porthouse, & Brooks, 2005; Xin & Jitendra, 1999). 


Conclusions 


For more than a century people have used theoretical assumptions to argue 
that general intelligence constrains decision making quality, causing 
substantial differences in human potential and outcomes. In turn, some have 
argued that such associations should partially dictate the structure of our 
policies, rights, institutions, and welfare practices. Many scientists have 
endeavored to map underlying issues in transparent ways. Many others have 
used these assumptions as justification for racist, sexist, and violent 
discrimination, disenfranchising millions of people and minority groups 


(Gould, 1996; Nisbett et al., 2012). Setting aside moral and ethical outrage, at 


the heart of the scientific issue is a basic question about the extent to which 
abilities like fluid intelligence actually constrain decision making quality. 
While theory of the past was built on broad assumptions about observed 
correlations, emerging experiments, training programs, and cognitive process 
tracing studies provide converging causal evidence. Contrary to long-held 
assumptions, skilled decision making does not generally require high levels 
of fluid intelligence or special abstract reasoning capacities. At an extreme, 
we find overwhelming evidence in decades of research on verifiable expert 
performers. If there are any effects of general cognitive abilities on expert 
decision-makers, they are increasingly trivialized and hard to detect 
compared to the profound decision quality benefits that accumulate with 
deliberate practice and the acquisition of specialized skills and knowledge 
(Ericsson et al., 2006). Even among non-experts, evidence indicates that 


decision making is an acquired skill that generally operates independently of 


fluid intelligence. One’s acquired level of statistical numeracy in particular, 
as measured by the Berlin Numeracy Tests and others, tends to be the single 
strongest predictor of general decision making skill across laboratory, 
naturalistic, and real-world contexts. This finding is consistent with the 
extensive evidence showing that quantitative skills are among the most 
influential educational variables associated with advancing economic 
prosperity in industrialized countries (Hanushek & Woessmann, 2010; Hunt 
& Wittmann, 2008). Results also reveal that simple interventions and brief 
training programs (e.g. visual aids and adaptive computerized tutors) can 
dramatically improve skilled decision making of diverse individuals who 
vary widely in abilities, proficiencies, education, backgrounds, values, and 
countries of residence (Garcia-Retamero & Cokely, 2011, 2013, 2017; see 
also Bruine de Bruin & Bostrom, 2013; Bruine de Bruin et al., 2007; 
Fischhoff, 2013; Fischhoff, Brewer, & Downs, 2012; Peters, 2017; Petrova et 
al., 2014, 2015; Trevena et al., 2013). 


It is noteworthy that statistical numeracy and other practical inductive 


reasoning tasks have historically been neglected in psychometric intelligence 
research, given that these are the primary factors linking quantitative abilities 
and general decision making skill. This robust association in part reflects 
shared elements on surface levels (e.g. numbers are common features of 
many decisions) and at foundations (e.g. understanding risk and uncertainty). 
Consistent with Skilled Decision Theory, statistical numeracy predicts skilled 
decision making because both require practical probabilistic reasoning and 
skilled metacognition (e.g. accurately evaluating and integrating thoughts, 
feelings, risks, and values). When these skills are combined with personally 
meaningful deliberation, they tend to promote a thorough and robustly 


representative understanding (e.g. adaptive mental models and integrated 


situation awareness), circumventing basic attentional capacity limitations 
with well-organized knowledge structures in long-term memory (e.g. vast 
personal knowledge and_ experience). Ultimately, a representative 
understanding of a decision problem helps people intuitively and precisely 
feel the relative weights of the essential issues, thereby enabling the adaptive 


use of heuristic evaluation and decision strategies. 


Understanding Risks 


Despite recent advances in our understanding of skilled decision making, 
several obstacles present great challenges. Strongly polarized beliefs and 
values exist in most communities and countries. Consistent with the current 
review, improvements in communications and clarifications of material facts 
are likely to resolve many of these issues. Nevertheless, research also 
indicates that some of these biases may never go away (Feltz & Cokely, 
2009, 2012, 2013, 2016). Some fundamental morally relevant biases are so 
robust they persist even among skilled reasoners, including verifiable experts 
who have devoted their lives to understanding the specific and relevant 
philosophical issues (Cokely & Feltz, 2009a, 2009b, 2014; Kahan, Jenkins- 
Smith, & Braman, 2011; Schulz, Cokely, & Feltz, 2011). In part, such biases 


persist because we don’t have reliable methods that can provide accurate 


feedback about the objective truth of many moral questions (if it even exists). 
We can precisely measure things like the weight of gold, but the same cannot 
be said for assessments of issues like rightness, justice, or equality. Even 
though skilled decision-makers can approximate normatively superior 
decision making standards, these standards cannot tell us which decisions we 
Should make unless we know what we should value. Moreover, decision 
sciences and technologies are rapidly enhancing the ability of some to control 
and manipulate our choices, often without our awareness or consent. While 
some valuable efforts are being deployed to help more of us make beneficial 


decisions without limiting our choices (Thaler & Sunstein, 2008), many 


others are designed to promote special interests and values via non-rational 


persuasion and choice architecture manipulation. 


As decision science advances we will increasingly be faced with a 
fundamental ethical question: Who should decide how we live our lives? A 
straightforward answer is that every person who is competent should have the 
opportunity to make their own decisions, given basic conditions (e.g. absent 
unwanted infringement on the autonomy of others). This perspective accords 
with widely accepted standards for ethical and informed decision making that 
emphasize autonomy and the importance of opportunities for deliberation in 
the light of one’s own values (Drane, 1984). If we aspire to satisfy these 
ethical standards and promote related democratic ideals, decision-makers will 
need a balanced but not necessarily extensive understanding of the risks and 
implications of various courses of action — i.e. a_ representative 
understanding (Feltz, 2015; Feltz & Cokely, 2016; Fischhoff et al., 2011; 


Garcia-Retamero & Cokely, 2017). Developing efficient scientific means of 


identifying and promoting representative understanding across evolving 
domains and conflicts is a formidable and worthy task. Methods developed 
for research on expert performance can help light the way for those decision 
scientists who dare to take on informed decision making challenges in 
complex, controversial, and high-stakes domains (e.g. climate change, 
energy, education, health, cybersecurity). To the extent that we improve 
representative understanding, the research reviewed here indicates that 
diverse people will be better prepared to make and discuss life-altering 
decisions in adaptive and ethical ways. Over time, even small improvements 
in skilled decision making tend to add up to substantial personal and societal 
benefits. It is a great gift and responsibility to know that nearly everyone has 
the ability to make well-informed and skilled decisions so long as they 


understand risks. 
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Introduction 


Most sports seasons culminate with an all-star game, which is a 
championship game of teams consisting of the best-performing athletes in the 
league. In addition to marking the end of the season, the all-star game can 
highlight the distinction between expert teams and teams of experts. At these 
elite levels, what sets teams apart and provides a distinct advantage to one 
team over the other is not how skilled the members are (they are all at the 
pinnacle of skill in their sport), but whether individuals can anticipate the 
moves of other team members, whether they can adapt to swiftly changing 
circumstances, and whether they can implicitly coordinate. That is, while all 
competitors in a league are inherently considered teams, those that are most 
successful are those that are adept at “teamwork.” In a similar vein, Olympic 
teams are formed of highly qualified athletes who must work as effective 
team members to win the gold. In 1992, the United States Olympic basketball 
team was termed “The Dream Team” due to the extraordinary basketball 
talent on the team (e.g. Michael Jordan, Magic Johnson), and their ability to 
defeat opponents by an average of 44 points per game. This team eventually 
proceeded to win the gold medal against Croatia. Yet in 2004 when the 
United States’ Olympic basketball team also housed some of the most 
talented players including LeBron James and Dwayne Wade, this team of 
experts was deemed one of the most disappointing Olympic teams after 
losing several games. It is clear from these examples that a team built of 
members with the highest technical prowess may propel the team to the 


highest levels of achievement; however, it is the well-developed “team- 


based” skills coupled with technical skill that represent the most effective 
teams. 

Another differentiating process between expert teams that are comprised 
of members with high technical skill (i.e. team of experts), rather than a 
combination of technical skill and teamwork skill (i.e. expert team) is that the 
former emerge compositionally (i.e. individual-level counterparts are similar 


in form to those at the team level; Kozlowski & Klein, 2000), while processes 


within a team of experts emerge in a compilational manner (i.e. team-level 
constructs manifest at a different level than individual-level counterparts). In 
other words, expert teams are more than just the sum of each individual team 
member’s level of experience. So what influences the processes within these 
teams to emerge in a compilational fashion? What are the contributing 
success factors to expert teamwork? 

This chapter addresses these questions by reviewing the teams literature 
over the past ten years. This updates the review of Salas and colleagues 
(Salas, Rosen, Burke, Goodwin, & Fiore, 2006), which described the 
conditions under which teams of experts become expert teams. The goal of 
the current chapter is to continue where this research left off in 2006 to 
provide an accurate depiction of current research and theory, and identify 
salient future directions for this work. 

According to Salas and colleagues (2006, p. 440), teams can be 


described as 


a set of interdependent team members, each of whom possesses unique 
and expert-level knowledge, skills, and experience related to task 


performance, and who adapt, coordinate, and cooperate as a team, 


thereby producing sustainable, and repeatable team functioning at 


superior or at least near-optimal levels of performance. 


This definition highlights the fact that not only are effective teams 
characterized by high levels of individual expertise (i.e. they have a high 
average level of expertise), but they are experts at team process and function. 
In order to have an expert team, the individual team members must possess 
expertise in their domain and expertise in contributing to and facilitating 
teamwork processes. The current chapter highlights what represents expert 
teamwork processing in order to identify how a team of experts can transform 


into an expert team. 


Chapter Aims and Organization 


Although the teamwork elements critical for expert team success have not 
significantly changed since the review conducted by Salas and colleagues 
(2006), researchers have developed new theories and created new models 
(e.g. the team adaptation nomological network model; Maynard, Kennedy, & 
Sommer, 2015) that aim to explain team phenomena. New mediating 
mechanisms and moderating variables of existing teamwork competencies, 
processes, and outcomes have been described. Moreover, since the Salas et 
al. (2006) review there has been an increase in the number of publications 
related to expert team performance. A search using the term “expert team” 
within Google Scholar yielded 14,800 results when including the years 
2006-16 in comparison to 5,440 results when restricting the years to 
1981-2006 (August, 2016). In this chapter we have deviated from the 
original seven areas used to frame the review by Salas and colleagues (2006). 
This is because the topics discussed in the current chapter are witness to more 
research advancements within the last ten years. 

We will first briefly review the theories prominent in previous decades 
and then outline new theoretical models and empirical evidence related to the 
antecedents and moderators of expert team performance. Specifically, we will 
focus on five critical considerations which include team competencies, 
emergent states, and characteristics. They include: team adaptation, shared 
cognition, team leadership, team composition, and the ability to overcome 
challenges associated with virtuality. Researchers have suggested that other 


aspects of teamwork are significant contributors to team effectiveness (e.g. 


monitoring progress, conflict management; LePine, Piccolo, Jackson, 
Mathieu, and Saul, 2008); however, based on the prevalence of the former 
competencies within the scientific and theoretical literature (e.g. Burke, 
Salas, Wilson-Donnelly, and Priest, 2004), we believe these five constructs 
are the most notable for team success, and expert team superiority. Following 
a discussion of each of the aforementioned constructs, we briefly outline the 
novel research methods that are being implemented to study expert teams. 
Finally, we discuss the state of the literature. Specifically, we review the 
heterogeneous ways in which team constructs are operationalized and 
measured in current research and what future research can contribute to help 
the field more accurately understand, quantify, and interpret the building 


blocks and processes of expert teams. 


Theories of Team Effectiveness: A Brief 
Review 


Over the past decade, hypotheses about team effectiveness have continued to 
be introduced, augmented, and improved. These hypotheses are applicable to 
a broad range of teams, including expert teams. However, the theories and 
conceptual models that sparked the initial expansion of team research remain 
prevalent in guiding research endeavors. Specifically, the input-process- 
output (IPO) model (Hackman, 1987; McGrath, 1984; Steiner, 1972) of team 


effectiveness continues to be referenced and has served as a foundation for 


subsequent theorizing. The IPO model specifies that a linear process 
determines team performance; this process is initiated by an input (i.e. 
resources, equipment) which facilitates a process (i.e. behaviors geared 
toward converting inputs to outputs). Finally, the process fosters an output 
(e.g. team performance). Ilgen, Hollenbeck, Johnson, and Jundt (2005) 
argued that this model should be expanded to include a mediator (i.e. IMOI 
model) in order to account for “the broader range of variables that are 
important mediational influences with explanatory power for explaining 


variability in team performance” (Ilgen et al., 2005, p. 520). Generally, 


subsequent models have taken this perspective, characterizing team 
performance as less of a linearly driven, static output and more of a cyclical, 
iterative pattern that influences subsequent performance levels. 

Moreover, a number of conceptual models that explain team processes 
and other aspects of teamwork (e.g. emergent states) have been developed 


over the past decade. Specifically, models of teamwork that are based upon 


team cognitions, behaviors, and attitudes remain a primary focus of most 
theoretical efforts (Salas et al., 2006). Team cognition or shared cognition, at 
a high level, encompasses a collective team awareness of each team 
member’s role, responsibilities, knowledge, skills, and ability levels as well 
as other integral factors such as team goals and objectives (Cannon-Bowers, 
Salas, & Converse, 1993). It has long been established that this emergent 
state, or dynamic team state that is influenced by numerous factors (e.g. 


contextual factors) (Marks, Mathieu, & Zaccaro, 2001), is strongly related to 


performance (e.g. DeChurch & Mesmer-Magnus, 2010a). It is posited that 
the dynamic team state allows teams to achieve high levels of performance 
without explicit communication because team members can rely on shared 
team knowledge to determine how to respond to any given environmental cue 
without utilizing cognitive resources to discuss actions with the team (Entin 
& Serfaty, 1999). 

Team activities encompass a broad range of constructs, such as 
leadership, communication, and coordination, which are each argued to 
positively influence team outcomes (Salas, Sims, & Burke, 2005). Leadership 
is posited to direct and organize team members’ interdependent actions such 
that they positively contribute to task completion (e.g. Burke et al., 200Ga). 
Coordination continues to be viewed as consisting of both explicit (e.g. 
talking aloud about approaches to task completion) and implicit (e.g. 
completing tasks in an interdependent fashion without discussing task 
approaches aloud) processes, which both contribute to high levels of team 
performance (Espinosa, Lerch, & Kraut, 2004). 

Similar to the behavioral and cognitive aspects of team performance, 
team affect research continues to be guided by the same conceptual models 


that have underscored team research previously. Constructs such as trust and 


cohesion are subsumed by this category. Cohesion refers to the collective 
attraction or desire of the team members to work with the team (e.g. 
Goodman, Ravlin, & Schminke, 1987). In line with theories explaining the 
role of team cognition and behavior, these models suggest that, as affective 
states such as cohesion increase, team members feel more strongly committed 
to the team and thus have a stronger motivation to achieve high levels of 
performance (e.g. Tekleab, Quigley, & Tesluk, 2009). Consequently, 
fostering effective team behaviors, cognition, and affective states is necessary 
for facilitating high levels of team performance. Theorizing relating these 
constructs to team performance continues to remain the backbone of team 
research, including research on expert teams (e.g. Barrick, Bradley, Kristof- 
Brown, & Colbert, 2007; Lemieux-Charles and McGuire, 2006; Lim & 
Klein, 2006; Mathieu, Maynard, Rapp, & Gilson, 2008). 


When Do Expert Teams Do Best? 


In their 2006 review, Salas et al. described several characteristics of an expert 
team based on a review of the empirical literature. Specifically, they 
identified that expert teams hold a shared mental model (SMM), a facet of 
shared cognition, such that they can anticipate each others’ needs through 
shared knowledge structures and an understanding of the shared goals and 
means of achieving those goals (Orasanu, 1990). Today, SMM is still a 
critical theoretical concept, and is being explored in many research and 


development projects (Maynard & Gilson, 2014; Xiang, 2015), medical 


teams (McComb, Lemaster, Henneman, & Hinchey, 2015), and human—robot 
teams (Adams, DeLoach, & Scheutz, 2014). 

Another characteristic of expert teams is that they optimize resources by 
learning and adapting (Salas, Rosen, & DiazGranados, 2010). Effective teams 
have been shown to be adept at reallocating functions and compensating for 
one another (Maynard et al., 2015). Expert teams have also been noted to 
consistently practice a cycle of pre-briefing and debriefing (i.e. facilitated or 
guided reflection geared toward learning, typically led by an external source; 
Fanning & Gaba, 2007). This cycle allows teams to establish and revise team 
goals through feedback and reflection. In fact, meta-analytic evidence 
suggests that debriefs improve team performance by 25 percent in the fields 
of medicine, aviation, and other high-stakes industries (Tannenbaum & 
Cerasoli, 2013; Tannenbaum & Goldhaber-Fiebert, 2012). Nonetheless it is 


important to note that briefing and debriefing alone are not enough to ensure 


high performing teams. Debriefs must be diagnostic, objective, oriented 


toward both individuals and the team as a whole, and guided by competencies 
that matter to team process and functioning. They must be facilitated by 
individuals who withhold judgment and foster a climate of openness, 
honesty, and reflection (Salas, 2008b). 

Expert teams have also been noted to define clear roles and 
responsibilities, which allows them to manage expectations (Bray & Brawley, 
2002; Poulton & West, 1999) and enables them to have a clear, valued, and 
shared understanding of the missions’ definitive vision and goals. Expert 
teams typically have strong leadership, whether it be centralized or 
decentralized (e.g. shared leadership; emergent leadership). Expert teams rely 
on leaders who consistently evoke trust, provide situational updates, 
incorporate mechanisms for self-correction, and foster overall teamwork. 

In the last ten years, research has explored the role of shared leadership 
(Wang, Waldman, & Zhang, 2014), leadership in virtual teams (Hoch & 


Kozlowski, 2014), leadership in multi-team environments (DeChurch et al., 


2011), and new avenues for researching team leadership (Burke, 


DiazGranados, & Salas, 2011). Shared leadership in particular is an emerging 


concept that is important in virtual project teams (Eisenberg, Gibbs, & 
Erhardt, 2016; Hoegl & Muethel, 2016). Moreover, shared leadership has 


been shown to contribute not only to team performance but also to both team 


and individual creativity, partly through the increased knowledge sharing that 
occurs as a result of shared leadership behaviors (Gu, Chen, Huang, Liu, & 
Huang, 2016). Gu et al. (2016) also found that shared leadership contributes 
to knowledge sharing when the team task is more interdependent. 

Expert teams have been shown to foster collective trust and a sense of 
team orientation and confidence. Recently, researchers have explored the role 


of trust in cross-cultural teams (Mach & Baruch, 2015), the development of 


swift trust in virtual teams (Zakaria, Yusof, Affendi, Muton, & Ab, 2015), as 
well as the curvilinear relationship between collective efficacy and team 
performance (Rapp, Bachrach, Rapp, & Mullins, 2014). 

Finally, expert teams have been shown to manage and optimize 
performance outcomes, and create mechanisms for team cooperation and 
coordination (Salas et al., 2006). Such mechanisms have recently been 
explored in surgical teams (Vashdi, Bamberger, & Erez, 2013), and have 
been shown to result in a reduced number of adverse events and shorter 
surgery durations. These outcomes are mediated by the increase in team 
helping behaviors and workload sharing. Moreover, research is exploring 
meso-level structures (e.g. team scaffolds) (Valentine & Edmondson, 2014) 
as coordinating mechanisms in temporary groups. 

It is evident that all of these identified characteristics are important 
today but because the nature of teams and the enactment of teamwork are 
changing, so too must the research on these topics. In the following section, 
we discuss research advancements and boundary conditions in the areas of 
team adaptation, shared cognition, team leadership, team composition, and 


virtuality. 


Team Adaptation 


Presently, organizations face highly competitive markets and continued 
change (Algesheimer, Dholakia, & Gurau, 2011). The nature of the modern 
workforce is constantly shifting in response to globalization, technology 
advancements, changing employee demographics, and a changing economy 
(Kapoor, 2011; Lyons, Ng, & Schweitzer, 2014). In order to contend with 
these demands, the criticality of effective teamwork and collaboration has 
increased (Luca & Atuahene-Gima, 2007), and in order for a team to become 
an expert team, successful adaptive performance is necessary. For example, 
diversity (e.g. ethnicity, age, gender) among workers is increasing at a rapid 
rate (Hite, Daspit, & Dong, 2015). Despite constant changes within the team, 
expert teams possess the ability to adapt to such changes in order to maintain 
effective teamwork processes and performance. Burke and colleagues (Burke, 
Stagl, Salas, Pierce, & Kendal, 2006b) describe it as an input-process- 
outcome (IPO) model of situation monitoring, plan formation, plan 
execution, and self-reflection. 

Team adaptation is a critical construct for understanding how expert 
teams operate. In 2015, Maynard and colleagues synthesized research on 
team adaptation over 15 years; in addition to reviewing the literature, they 
have proposed a model of team adaptation. The authors describe team 
adaptation as a nomological network and explain that team adaptability (i.e. 
“the capacity of a team to make needed changes in response to a disruption or 


trigger”; Maynard et al., 2015, p. 655) leads to team adaptation processes (i.e. 


“adjustments to relevant team processes in response to the disruption or 


trigger giving rise to the need for adaptation”; p. 656). In turn, team 
adaptation processes engender team mediators (e.g. communication, 
cognition), which then lead to team adaptive outcomes (e.g. improved 
performance in novel team situations, recovery from team errors, improved 
decision effectiveness). In other words, the initial driving factor of effective 
team adaptation is the team’s inherent ability to modify, adjust, and alter 
behavior in response to current needs (i.e. team adaptability). 

This model can be illustrated by an empirical investigation of expert 
trauma teams. Kahol, Vankipuram, Patel, and Smith (2011) observed ten 
Level 1 trauma cases, and found that deviations (i.e. non-compliance with 
standard methods) by expert teams were classified as innovations, whereas 
deviations by novice teams were considered errors. In other words, due to 
their adroit adaptability efforts, expert teams are perceived as being better 
able to respond to and overcome novel, continuously changing environments 
and do so in an innovative and creative yet effective way. 

In addition to Maynard and colleagues’ (2015) model of team 
adaptation, researchers have also identified the role of team adaptation as a 
mediator in the relationship between team learning and team performance 


(Santos, Passos, & Uitdewilligen, 2016). Santos and colleagues (2016) drew 


on a prevalent theoretical argument that suggests team members engage in 
learning processes via evaluating past instances of performance and 
developing new strategies to improve; this argument further suggests that this 
process of evaluation and correction results in team adaptation and, 
consequently, positive outcomes such as performance (Burke et al., 2006b; 


Kozlowski & Bell, 2008; Rosen et al., 2011). Team adaptation as a process is 


a critical element that enables team members to learn, as evidenced by Santos 


et al.’s (2016) results demonstrating that team adaptation mediates the 


relationship between team learning and performance. Ultimately, these 
findings suggest that team adaptation may enable team members to acquire 
expertise through learning. 

Researchers have also leveraged Structural Adaptation Theory (SAT; 
Johnson et al., 2006) to understand expert teams. According to SAT, social 
systems (e.g. teams) are similar to physical systems such that they can be 
categorized by their degree of complexity; the more complex a social system 
is, the more resources are needed to maintain its structure. Over time, these 
complex and organized social systems will become simpler and more chaotic. 
As such, without formal interventions, complex systems that are highly 
structured (e.g. hierarchical, specialized, collective) will become disordered 
and chaotic (e.g. decentralized, undifferentiated, individualistic). As a result, 
in order to promote successful team adaptation, it is necessary for 
organizations to implement cooperative reward structures and reinforce a 
collective culture. 

An adaptive performance measurement system is also critical in 
fostering adaptation. A successful adaptive performance measurement system 
can be created by following the theoretically based principles outlined by 
Rosen et al. (2011), which were developed from a review of the empirical, 
theoretical, and methodological literatures on team adaptation, performance, 
and measurement. SAT theory has been empirically supported and well 
validated (e.g. Beersman et al., 2009; Johnson et al., 2006) and continues to 


be a foundation for understanding why expert teams succeed. 


Shared Cognition, Shared Mental Models, and 
Transactive Memory 


Another characteristic of expert teamwork is the team’s ability to achieve a 
shared understanding of the task, the indivuals’ roles, and the action plan 
needed to achieve team goals. In the literature this has been termed “shared 
cognition.” Team members within an expert team are able to coordinate with 
one another to meet complex goals, and they can sometimes do so without 


overt communication (Rico, Sanchez-Manzanares, Gil, & Gibson, 2008). As 


these team members do not exhibit the need for explicit communication, not 
only are they able to perform effectively, but they also do so efficiently 


(Cannon-Bowers & Bowers, 2011). That is, they are able to complete tasks 


within a reduced time frame without committing errors. Nonetheless, 
communication is a noted critical component of teamwork (Salas, Rosen, 


Burke, & Goodwin, 2009b). So how do expert teams effortlessly perform 


without it? According to empirical research, it is because they have a high 


degree of shared cognition (e.g. DeChurch & Mesmer-Magnus, 2010b). 


As defined above, shared cognition represents a mutual understanding 
about team-related concepts between team members (Rosen, Furst, & 


Blackburn, 2006). Generally, the term shared cognition is used within the 


literature to represent multiple constructs, subsuming different types of 
cognition that are conceptualized and measured in distinct manners. 


Specifically, transactive memory systems (TMS) and shared mental models 


(SMM; DeChurch & Mesmer-Magnus, 2010b) are the two most commonly 


studied forms of shared cognition. Although there is some overlap among 


these constructs, they represent distinct conceptualizations of shared 
cognition and the implications of findings associated with research pertaining 


to each should be considered separately. 


Distinguishing between TMS and SMM 


DeChurch and Mesmer-Magnus (2010b) suggest that the most fundamental 
distinction between SMM and TMS is that SMM encompasses _ the 
knowledge shared among all team members whereas TMS represents the 
manner in which knowledge is distributed among team members. More 
specifically, TMS can be defined as “the cooperative division of labor for 
learning, remembering, and communicating relevant team knowledge” 
(Lewis, 2003, p. 587). It is generally evaluated via a self-report measure 


created by Lewis (2003), which includes three dimensions, consisting of 


specialization (i.e. the knowledge of where expertise lies within the team), 
credibility (i.e. trust in the reliability of this expertise), and coordination (i.e. 
the coordinated use of this expertise). These components are generally 
averaged to create one aggregate score to represent the entire team. Although 
this represents the most common form of study, some researchers have 
attempted to measure this construct in a different manner. Austin (2003) used 
several measures to evaluate TMS, such as providing a problem statement 
requiring team members to indicate which team member had the most 
appropriate expertise for different scenarios. Accuracy was calculated by 
“matching group member identification of experts with self-report ratings of 
expertise” (Austin, 2003, p. 870). 

Regardless of measurement approach, on the whole, both meta-analytic 


(DeChurch & Mesmer-Magnus, 2010b) and empirical (e.g. Smith-Jentsch, 


Kraiger, Cannon-Bowers, & Salas, 2009) studies indicate that when a team 


has a strong TMS, team processes become more efficient. Specifically, 


Austin (2003) found empirical evidence that TMS predict team performance 


when combined together and, when parsed apart, TMS accuracy has the 
greatest impact on performance. Although few subsequent studies have 
sought to utilize measures beyond the self-report measure created by Lewis 
(2003), this work suggests that there may be value in incorporating additional 
measures. 

Conversely, SMM are defined as the shared “knowledge structures held 
by members of a team that enable them to form accurate explanations and 
expectations for the task, and in turn, to coordinate their actions and adapt 


their behavior to demands of the task and other team members” (Cannon- 


Bowers et al., 1993, p. 221). The content of SMM and the manner in which 
they are measured may vary widely across studies. Several common 
approaches to measurement include card sorting, concept mapping, and 
pairwise comparison ratings. Card sorting, for example, might involve having 
the participants sort examples of teamwork into various categories (e.g. 
Smith-Jentsch, Campbell, Milanovich, & Reynolds, 2001a). Concept 
mapping involves participants categorizing terms into a structure, which may 
be predetermined or created by the participants themselves, that represents 
the relationship among the different terms (e.g. Marks et al., 2000). Pairwise 
comparison generally entails rating the similarity of various concepts. 
However, Smith-Jentsch (2009) cautions that even within these different 
types of measures, there may be variations. Moreover, researchers may also 
choose to calculate accuracy (i.e. whether participants respond in a manner 
that aligns with experts), or similarity (i.e. whether participants respond in a 
manner that is consistent with their teammates), or both. 

In addition to measurement, the content of SMM may also differ. 
Researchers have examined various types of SMM, but these shared 


knowledge structures always refer to knowledge shared among all team 


members. For example, Cannon-Bowers and colleagues (1993) identified 
four types of SMM including equipment, task, team member roles and 
responsibilities, and team members. Conversely, DeChurch and Mesmer- 
Magnus (2010b) categorized SMM into task- and team-related mental 
models. Interestingly, meta-analytic results suggested that team mental 
models were more related to team processes while task mental models were 
more related to performance outcomes (DeChurch & Mesmer-Magnus, 
2010b). Shared team interaction mental models, or SMM encompassing 
knowledge related to “how team members should work together on a task 
within a given task domain, including information about who should do what 


at particular points in time” (Marks, Sabella, Burke, & Zaccarro, 2002, p. 5), 


have also been shown to increase team processes such as supporting behavior 
and coordination. Specifically, Marks and colleagues (2002) found that cross- 
training led to shared team interaction mental model development which, in 
turn, led to supportive behavior and coordination; these processes then led to 
enhanced performance. Smith-Jentsch, Cannon-Bowers, Tannenbaum, and 
Salas (2008) examined SMM encompassing elements of teamwork and found 
that teams trained with a guided self-correction team training program 
developed more accurate SMM which led them to perform better. 

Given the diverse ways in which SMM have been conceptualized and 
measured in the teams literature, it is important to review the manner in 
which studies operationalize and measure a form of SMM. before 
generalizing findings to the broader literature on shared cognition. However, 
we suggest that on the whole, the literature indicates that SMM contribute to 
enhanced team processes and team performance. The content of an SMM 
may affect performance via various mechanisms, but we argue that the 


ultimate effect (i.e. enhanced performance) is the same. We further suggest 


that expert teams require a high degree of shared cognition in the form of 
both TMS and SMM. 


Current Research on Shared Cognition 


Initial research on SMM and TMS generally focused on their relationship to 
team processes and performance; it also primarily focused on content 
pertaining to team- and task-related knowledge. However, current research is 
expanding to focus on shared cognition encompassing additional content. For 
example, Mohammed and Nadkarni (2014) recently studied how shared 
cognition pertaining to time might affect team performance. Specifically, 
Mohammed and Nadkarni (2014) described shared temporal cognition, or the 
“common understanding of the time-related aspects of executing collective 
tasks” (p. 405) and temporal TMS, or “the set of unique knowledge domains 
held by specific members combined with a shared understanding of who 
knows what and when that knowledge is needed in the team” (p. 405). They 
argued that these constructs have a unique influence on team performance 
through polychronicity, or team members’ combined preference for working 
on multiple tasks at once (Bluedom, Kalliath, Strube, & Martin, 1999). They 
found that, indeed, shared temporal cognition mitigated the negative impact 
of diversity in team member polychronicity on performance whereas 
temporal TMS worsened this effect. 

Additional work has focused on even more ways in which SMM can 
vary. Specifically, one of the criticisms of the way that SMM have been 
conceptualized is that they are considered to be solely based on deliberate and 
conscious processes (Salas et al., 2010). More recent work has identified that 
SMM can be conceptualized on a reflective level or a reflexive level, such 
that a SMM can develop through reasoning and deliberation or through a 


more automatic, intuitive, and affective manner (Healey, Vuori, & 


Hodgkinson, 2015). The authors argue that for teams to function effectively it 
is not sufficient to rely solely on deliberative mental models of the task but 
rather they must also share cognition at the reflexive level, due to the 
potential for illusory concordance and their potential to inflict conflicting 


influences on team behaviors (Latham, Stajkovic, & Locke, 2010). Similarly, 


Sonesh and colleagues (Sonesh, Rico, & Salas, 2013) argue that illusory 
shared cognition can impact team decision making. As such, it is necessary 
for expert teams to work on ongoing rather than static “interactive team 
cognition” (Cooke, Gorman, Myers, & Duran, 2013) whereby cognition is 
considered an activity rather than a property, is inextricably tied to context, 
and both implicit and explicit cognitive similarity is achieved and managed 
within a team. 

This notion of illusory concordance in SMM is buttressed by Santos et 
al.’s (2016) findings which suggest that sharing an inaccurate mental model 
in a team leads to a decrease in learning behaviors, which the authors 
interpret as representing a team’s “closed mind.” As such, it is necessary for 
researchers to continually update their perspective on the sub-facets of shared 


cognition and its potential counterintuitive and unintended effects. 


Team Leadership 


Team leadership, also referred to as shared leadership, has consistently been 
identified as an influential team behavior that has an impact on both team 
processes and outcomes across a wide range of teams (e.g. Salas et al., 2005; 
Wildman et al., 2012). Much of the initial work conducted on this construct 
stemmed from individual level leadership, but research has since expanded to 
focus on uncovering the underpinnings of team leadership. Team leadership 
has been defined as the “ability to direct and coordinate the activities of other 
team members, assess team performance, assign tasks, develop team 
knowledge, skills, and abilities, motivate team members, plan and organize, 
and establish a positive atmosphere” (Salas et al., 2005, p. 560). In other 
words, this team behavior has generally been conceptualized as any behavior 
directed at coordinating team members to facilitate task completion. 

Day, Gronn, and Salas (2006) reviewed the literature and noted that the 
study of “team leadership as a discipline appears to be on the cusp of some 
truly significant breakthroughs” (p. 211). Indeed, the past decade has seen a 
rapid surge in the volume of research conducted on team leadership, 
illuminating the impact of this team process on multiple variables. For 
example, Burke and colleagues (2006a) conducted a meta-analysis on shared 
leadership and found that it demonstrated a moderately strong relationship 
with team effectiveness outcomes. Moreover, it was also demonstrated to be 
strongly related to team learning. Thus, effective team leadership is one 
factor that may set apart an expert team from other teams. Muethel and Hoegl 


(2013) suggested that shared leadership in such teams should be studied using 


a social exchange theory perspective, which implies that individual behavior 
is driven by rewards from others (Blau, 1964; Cropanzano & Mitchell, 2005; 
Homans, 1958). Specifically, they posited that if team members jointly 
attempt to influence and accept influence from other team members, team 
leadership will be at its most effective. Others have suggested that shared 
leadership is especially integral in virtual project teams, as it may mitigate 
some of the issues associated with communication via virtual tools (e.g. time 
delay in correspondence) (e.g. Barnwell, Nedrick, Rudolph, Sesay, & Wellen, 
2014; Hoch & Kozlowski, 2014; Jawadi, 2013). Moreover, Mehra, Smith, 
Dixon, and Robertson (2006) found that different shared leadership structures 


may result in varying levels of team performance, noting the importance of 
distinguishing between how leadership is distributed throughout the team. 
However, despite these advancements, this area of research is still 
nascent. One problem relating to the study of shared leadership is a lack of 
construct clarity, as with shared cognition. In recent work, varying 
conceptualizations have been used to drive research initiatives. For example, 
Wang et al. (2014) recently conducted a meta-analysis on shared leadership. 
Their conceptualization of shared leadership specified that it consisted of the 
same behaviors as hierarchical leadership (i.e. team members leading each 
other toward goal completion) but differed because it is shared among team 
members. They thus examined multiple forms of shared leadership, including 
new-genre leadership (i.e. leadership that emphasizes transformational 
leadership), traditional forms of leadership (i.e. leadership that emphasizes 
status quo), and cumulative, overall shared leadership (i.e. leadership 
aggregated to the team level). Conversely, Hoch and Kozlowski (2014) 
conducted a study that measured shared leadership via behavioral, affective, 


and cognitive constructs, suggesting that shared leadership is behaviorally 


and qualitatively different from individual level leadership. On the whole, 
however, the literature suggests shared leadership is one element that 


separates expert teams from others. 


Team Composition 


Team composition, or the configuration of member attributes in a team 
(Levine & Moreland, 1990), has been argued to play an influential role in 
team processes and team performance (Kozlowski & Bell, 2003). There are 
many different characteristics upon which a team can be composed (e.g. 
personality, age, functional expertise); correspondingly, research has studied 
team composition in a variety of ways. For example, one salient 
conceptualization differentiates between surface-level (i.e. composition 
variables that are immediately apparent, such as age) and deep-level (i.e. 
composition variables that are not immediately apparent, such as personality) 
composition (Bell, 2007). Adding to this categorization, team composition 
research also varies in terms of its configural operationalization. As team 
composition represents a team-level variable, individual-level data must be 
aggregated to the team level in some fashion. The _ configural 
operationalization refers to the method by which the construct is aggregated. 
The most common methods of configural operationalization include 
minimum levels, maximum levels, variance, and average. As noted by Bell 
(2007), these variations illuminate why there is some inconsistency within the 
team composition literature. To help clarify how these different variables and 
varying configural operationalizations influence performance, Bell (2007) 
conducted a meta-analysis on deep-level composition variables. One of the 
most impactful findings indicates that “the best operationalization was 
dependent on the specific team composition variable of interest” (p. 607). For 


example, she found that the minimum level of agreeableness was most 


strongly related to performance (in comparison to other configural 
operationalizations of agreeableness) but the majority of composition 
variables were most strongly related when the average was utilized. 

Similarly, Tekleab and Quigley (2014) found that diversity in 
personality characteristics influenced team relationship conflict, and team 
member affective reactions. Specifically, their results suggest that low 
variance (i.e. similarity) in team member agreeableness, conscientiousness, 
and emotional stability weakens the relationship between conflict and 
affective reactions. However, high variance (i.e. heterogeneity) in the 
extraversion trait among team members also weakens the relationship 
between conflict and affective reactions. Post (2012) also explored team 
composition of two different types of thinking styles in a team. The study 
found that average sequential thinking (i.e. using logical thinking routines) as 
opposed to average connective thinking (i.e. linking many previously 
unconnected ideas), helps improve team innovation through cooperative 
learning. Thinking style affects team innovation though psychological safety 
and cooperative learning. 

Bell and colleagues (Bell, Brown, Abben, & Outland, 2015) reviewed 


the implications of team composition on space exploration missions. They 


suggest that because space crew teams, which can be assumed to represent 
expert teams, will have to work and live in extreme environments 
characterized by prolonged confinement, isolation, and communication 
delays, team composition is an important consideration for maximizing team 
processes and performance outcomes. They conclude that there is a need to 
identify ways to foster unit-level social integration within diverse crews and 
emphasize the importance of separating the effects of specific team 


composition variables to determine the traits (e.g. personality, values) that are 


associated with particular challenges and barriers to performance. This also 
highlights the need for clarity and guidance in terms of configural 
operationalization. 

Team composition has also been recently cited as a predictor of team 
creativity. Somech and Drach-Zahavy (2013) investigated team innovation as 
a function of team composition. They examined teams composed of 
functionally heterogeneous team members with creative personalities, and 
found that average individual creativity coupled with functional heterogeneity 
promotes team creativity. In this instance, functional heterogeneity was 
measured with a diversity index (Blau, 1977) which reflects how many 
different types or categories there are in a dataset, and simultaneously takes 
into account how evenly individuals are distributed among those types. It was 
found that a team composed of members with a high degree of functional 
heterogeneity is especially effective when the climate for innovation in the 
organization is high. 

While it is evident that composition influences team performance, 
research suggests that the effects of composition may also depend on the 
stage of the team’s development. Mathieu and colleagues (Mathieu, 


Tannenbaum, Donsbach, & Alliger, 2014) reviewed four team composition 


models and discussed the potential for different periods within a performance 
episode or team’s development, to influence the saliency of different 
compositional mixes of a team. They present a temporal framework and 
optimization algorithm by integrating all four composition models that 
suggest when membership change is necessary, and synthesize the previously 


incoherent and disjointed literature on team composition. 


Virtuality 


Research has further indicated that team virtuality is a team characteristic that 
can influence the effectiveness of team processes and outcomes (Wildman et 
al., 2012). Virtuality in organizations is growing (Perry, 2008). A 2012 
survey suggests that approximately 60 percent of multinational organizations 
use virtual teams (Society for Human Resource Management, 2012), and that 
number is expected to grow. Team virtuality is defined as the degree to which 
team members are separated by spatial and temporal boundaries (Townsend, 
DeMarie, & Hendrickson, 1998). As a result, virtual teams must rely on 
electronic tools to collaborate and coordinate on various aspects of task 
completion (Gibson & Cohen, 2003; Townsend et al., 1998). For example, 
teams with a high degree of virtuality might solely rely on email to 
communicate about task processes. However, due to the inherent time lags 
associated with technology-mediated information exchange, 
misunderstandings, lost communication, and a reduction of information- 
seeking attempts all influence virtual team performance (Andres, 2012). 
Virtuality has been defined in numerous ways, but was _ initially 
conceptualized as a dichotomous variable. Researchers suggested that teams 
fully relied either on virtual tools or on face-to-face communication to 
coordinate and engage in other team processes (e.g. Guzzo & Dickson, 1996). 
However, researchers have more recently begun conceptualizing virtuality as 
a continuous variable. For example, Kirkman and Mathieu (2005) describe a 
framework which specifies that virtuality consists of three dimensions, 


namely the extent of reliance on virtual tools, synchronicity, and 


informational value, and each of these dimensions is described as falling on a 
continuous scale. The extent of reliance on virtual tools refers to how much 
the team members utilize virtual tools to communicate. For example, if team 
members primarily rely on face-to-face communication, they would be 
classified as low on this dimension. Synchronicity is defined as the extent to 
which the virtual tools being utilized for communication can be utilized in 
real time as opposed to with a time lag. Finally, informational value refers to 
the richness of information the virtual tools are capable of conveying. This 
framework has been adapted and used to inform work uncovering the 
influence of virtuality on team processes, illustrating the importance of 
conceptualizing virtuality as a continuous variable. For example, Mesmer- 
Magner, DeChurch, Jimenez-Rodriguez, Wildman, and Shuffler (2011) meta- 
analytically examined the moderating influence of virtuality on the 
relationship between team information sharing and performance. They found 
that this relationship was complex and uncovered effects that would have 
been hidden if they had defined virtuality in a dichotomous manner. In 
particular, their results indicated that virtuality is associated with enhanced 
unique information sharing (i.e. the sharing of information that has not 
previously been distributed throughout the entire team) but decreases the 
openness (i.e. the sharing of information that is both unique and commonly 
held) of information sharing. Moreover, the necessity of different forms of 
information sharing was contingent upon the team’s level of virtuality. 

More recently, Hoch and Kozlowski (2014) have suggested that 
virtuality is multifaceted and in addition to geographic dispersion (the relative 
amount of face-to-face and electronic media usage; Kirkman, Rosen, Tesluk, 
& Gibson, 2004), it should also include cultural diversity (Hinds, Liu, & 
Lyon, 2011). Hoch and Kozlowski’s (2014) empirical findings suggest that 


cultural differences among members of a team can add to the level of 
virtuality within a team, due to their adverse effects on team identification 
(Au & Marks, 2012). Other researchers have sought to uncover the impact of 
virtuality on other team processes. For example, Kanawattanachai and Yoo 
(2007) found that virtuality can hinder the rate at which shared cognition 
develops. Conversely, Hoch and Kozlowski (2014) demonstrated that 
virtuality weakened the relationship between hierarchical leadership and team 
performance but simultaneously strengthened the relationship between 
structural supports (i.e. reward systems, communication and information) and 
team performance. Moreover, Lowry and colleagues (Lowry, Roberts, 
Romano, Jr., Cheney, & Hightower, 2006) found that virtual teams can be 
more effective in overcoming challenges associated with large team sizes 
than face-to-face teams. This is in part because of the tools and resources 
inherent to virtual interactions that impact the social and structural networks 
of the team. Bryant, Albring, and Murthy (2009) found that certain forms of 
computer mediated communication (CMC) reduce social loafing by team 
members. Moreover, Suh and colleagues (Suh, Shin, Ahuja, & Kim, 2011) 
found that personalized CMC, such as email and instant messaging, had a 
positive effect on extra-group network size and structural holes; conversely, 
communal CMC, such as group discussions, and audio conferences, 
increased intragroup tie strength. 

The effects of virtuality on performance have several mediators and 
moderators that have been identified (Gilson & Shalley, 2004). For example, 
leadership has been noted as an important moderator of the virtuality—team 
performance relationship. Henderson’s (2008) findings suggest that in virtual 
teams it is important for team members to be satisfied with their leader, as 


that helps them understand and “decode” messages from leaders who are 


geographically distant. Similarly, Hoch and Kozlowski (2014) suggest that 
virtuality can weaken the relationship between leadership and performance. 
These results suggest that virtual team leaders must perform at a particularly 
high level to compensate for the attenuating effects of virtuality. 

Virtuality’s effect on team performance is also mediated by conflict 
management. As geographical distance increases within a team, conflict 
management skills weaken (Cramton & Hinds, 2005). This is an important 
mediator to consider because conflict is considered to be more likely in 


virtual teams than in co-located teams (Furumo, 2009), which ultimately 


affects team satisfaction (Stark & Bierly, 2009). As such, leaders must ensure 
that they work to reduce conflict in highly virtual teams (Wakefield, Leidner, 
& Garrison, 2008). Leaders must also work harder to maintain high levels of 
coordination. Pennarroja and colleagues (Penarroja, Orengo, Zornoza, & 
Hernandez, 2013) found that as virtuality increased, team coordination 
decreased. However, this relationship was mediated by levels of trust. As a 
result, team leaders should strive to ensure high levels of trust among 
members of virtual teams to maintain coordination. In fact, it has been 
described as the “glue” that keeps virtual teams together (Crisp & Jarvenapaa, 
2013). 

Recent research has also emphasized the important role that SMM 
(Andres, 2012), and TMS (O’Leary & Mortensen, 2010) have on the 


performance of virtual teams. Emergent states like efficacy (Ortega, Sanchez- 


Manzanares, Gil, & Rico, 2010), team commitment (Lin, Chiu, Joe, & Tsai, 
2010), cohesion (Jarman, 2005), team identity (Au & Marks, 2012), and 
psychological safety (Maynard, Mathieu, Rapp, & Gilson, 2012) have all 


been noted as important mediators of the virtuality-team performance 


relationship. 


Taken as a whole, current understanding pertaining to the impact of 
virtuality on various team processes and outcomes suggests that the effects 
are complex. Thus, the moderating influence of virtuality should be 


investigated on a case-by-case basis. Gilson and colleagues (Gilson, 


Maynard, Young, Vartiainen, & Hakonen, 2015) outlined several areas where 
future research on virtual teams is most needed, noting that future work 
should seek to explore familiar, intact teams in a variety of field settings, as 
previous work in field settings has focused primarily on knowledge-intensive 
workers. We also advocate for future work exploring the influence of 
virtuality in novel settings. Specifically, we suggest that expert teams will 
contend with varying degrees of virtuality differently than other teams and, to 
our knowledge, little work examining this variable within expert teams has 
been conducted. Consequently, research should seek to examine how the 
relationships among integral team processes and team performance are 


influenced by virtuality in expert teams. 


Current Methods Used to Investigate Expert 
Teams 


Team measurement methods are critical to accurately understanding the 
dynamics within a team. Existing methods for measuring expert teams 
include observations in the field (Lipshitz, 2005), simulations (Rosen et al., 
2008; Woods & Sarter, 1993), self-report Jung & Sosik, 2002), and peer- 
report (Valentine, Nembhard, & Edmondson, 2015) methods; however, many 


of these traditional methods have been criticized for their potential for biased 
and inaccurate ratings due to human error, distraction, social desirability, halo 


effects, and cognitive overload, among other biases (Lipshitz, 2005). In fact, 


Valentine et al. (2015) found that of 39 available self-report surveys 
measuring teamwork in medical teams, only 10 met criteria for psychometric 
validity, and only 14 showed significant relationships to non-self-report 
outcomes. 

To address these many shortcomings of traditional measures of expert 
teamwork, researchers have begun to conduct empirical studies that use 
physiological sensing and wearable computing technology to understand and 
predict high-level behavioral constructs such as affect, activity, and creative 
output (Burleson, Picard, Perlin, & Lippincott, 2004; Choudhury & Pentland, 
2003; Kapoor, Burleson, & Picard, 2007; Kim, Chang, & Pentland, 2007; 
Olguin Olguin, 2007; Olguin Olguin, Gloor, & Pentland, 2009a, 2009b; 
Olguin Olguin et al., 2009c). It has been found that affective computing has 


been able to distinguish individuals’ affective state with 81 percent accuracy 


(Kapoor et al., 2007). This has implications for team measurement as well 


(Henning, Boucsein, & Gil, 2001). In a study of entrepreneurial teams at 
MIT, the best predictor of a “winning” team was the average percentage of 
speaking time across team members, followed by physical activity level, 
speech energy, and time in close proximity to each other (Olguin Olguin & 


Pentland, 2010). In the field of healthcare, it has been found that it is possible 


to predict variation in patient length of stay as a result of nurses’ physical 
activity and face-to-face interaction time within a team (Olguin Olguin & 
Pentland, 2010). 

These findings suggest that there is great potential in measuring 
physical, proxemic, and kinetic indicators of teamwork and other “soft” skills 
that are often overlooked and unmeasured. However, despite these indicators, 
teamwork is often rated poorly due to a perceived lack of explicit 
communication. For a more accurate and complete assessment of teamwork 
in expert teams, it is critical to take a more holistic approach and capture the 
indicators of teamwork that are often unseen, or unmeasured. For example, 
an important aspect of social behavior is the study of non-linguistic signals 
during face-to-face interactions. According to Heibeck and Pentland (2008), 
these signals can be measured by analyzing the timing, energy, and 
variability of speech and body movement patterns. Pentland (2008) describes 
four different types of signals in humans: influence (the extent to which one 
person causes the other person’s pattern of speaking to match their own 
pattern), mimicry (the reflexive copying of one person by another during a 
conversation), activity (speaking time and physical activity level), and 
consistency (low variability in the speech signal or physical activity). 

Moreover, non-verbal signals such as movement patterns provide 
valuable insight into team and social dynamics. That is, where are team 


members looking? Where are they moving? Who are they making eye contact 


with? How hurried is their pace? How, rather than what, are they 
communicating? What are the physiological and linguistic characteristics of 
team members that can help illuminate the existence of expert teamwork vs. 
novice teamwork? The idea is that unobtrusive measures may provide 
incorrect information about the team’s functioning, especially if the 
communication is implicit. In expert teams, good teamwork is often “quiet,” 
indicating “implicit” coordination that does not require explicit 


communication (Volpe, Cannon-Bowers, Salas, & Spector, 1996), which 


would require measuring other more objective factors. Rosen and colleagues 
(Rosen, Dietz, Yang, Priebe, & Pronovost, 2014) review sensor-based team 
measurement in the field of healthcare and put forth a framework for the 
development of a multi-method team performance measurement system. 
However, to date no one has synthesized the specific physiological, kinetic, 
and linguistic features that indicate the existence of “expert” teamwork. What 
is needed is a foundation upon which valid assessment of these features can 
be carried out to inform and supplement team science, and particularly 


teamwork in expert teams. 


The Next Ten Years: Opportunities for Future 
Team Research 


This chapter has reviewed theories that explicate the factors that differentiate 
expert teams from other teams. Specifically, theorizing related to team 
behaviors, affect, and cognition were reviewed and can continue to be 
utilized to inform the development of future theoretical models and 
frameworks. However, more recent work has focused on uncovering more 
nuanced details of these frameworks, such as identifying the impact of 
moderators and how they interact to influence team functioning. Future work 
should continue to further this trend such that team performance can be 
predicted within a variety of conditions. Moreover, it is important to better 
understand the dynamics of teams over time. In fact, several researchers have 
recently conducted longitudinal studies that explore the role of team 
cohesion, and collective task engagement in creativity tasks in predicting 
team performance over time (Rodriguez-Sanchez, Devloo, Rico, Salanova, & 
Anseel 2016). 

Similarly, Quigley (2013) has shown that leadership efficacy can change 
over time and should thus be measured from multiple levels and time points. 
His findings suggest that leader extraversion and cognitive ability are 
predictors of initial levels of leadership efficacy (i.e. confidence in one’s 
leadership abilities), but that personality traits like agreeableness, emotional 
stability, and openness are predictors of the change over time in leadership 


efficacy. 


These longitudinal results have many implications for how we study and 
understand the emergence of shared leadership in teams. Future work should 
continue to conduct large-scale, multilevel, longitudinal studies to uncover 
more granular characteristics, processes, and activities of expert teams and 
how they operate dynamically. Collecting continuous data on modes of 
communication including emails, phone logs, cloud-based shared resources, 
and other channels may be one avenue through which this can be achieved. 

One of the most pressing issues to ensure the impact of the next decade 
of teams research is to avoid construct confusion by clarifying the language 
utilized in the theorizing. In the next section we present opportunities for 
construct de-confliction and disambiguation, whereby we discuss what might 
be done to address the difference between related constructs within the 


critical team considerations presented in this chapter. 


Construct De-confliction 


As mentioned throughout the chapter, there is a need to reach a consensus 
regarding the definition of each of the theoretical constructs referenced in this 
chapter and their underlying components. This represents a primary direction 
for future research. For example, regarding shared leadership, researchers 
have conceptualized it as consisting of the same behaviors as individual level 
leadership, such as transformational leadership, albeit at the team level (e.g. 
Wang et al., 2014). However, others (e.g. Hoch & Kozlowski, 2014) have 
suggested that behaviors that define shared leadership are behaviorally 


distinct from those that classify as individual level leadership. Each form of 
team leadership may have a unique and differential impact on team processes 
and outcomes; thus, future researchers should seek to more clearly categorize 
these constructs and determine how they relate to one another and the overall 
construct of shared leadership to determine more definitively how this 
process functions in expert teams. 

A review recently completed by Kozlowski, Mak, and Chao (2016) 
provides one helpful categorization scheme that may be used to evaluate or 
conduct shared leadership research, suggesting that the literature can be 
broadly organized by theory. The authors describe four main approaches to 


studying this construct: 


1. Transformational leadership (i.e. leaders seek to encourage and 


inspire their followers). 


2. Leader—member exchange (i.e. leaders are assumed to develop unique 


relationships with each follower). 


3. Shared leadership (i.e. leadership is distributed among the team). 


4. Functional leadership (i.e. leaders focus on how core functions are 


influenced by team and task variations). 


We advise using this approach to interpret the research on shared 
leadership. It also represents one potential method for comparing the relative 
effectiveness of different forms of leadership. 

Regarding shared cognition, we suggest that researchers consider the 
literature on TMS and SMM separately, as they represent two distinct forms 
of shared cognition. We also argue that the content of the SMM being studied 
should be taken into careful consideration before results are generalized. For 
example, the meta-analytic findings completed by DeChurch and Mesmer- 
Magnus (2010b) suggest that team and task SMM influence performance in 
different manners, as previously discussed. In general, the literature indicates 
that SMM encompassing team content (e.g. shared knowledge related to team 
member roles) may affect performance via enhancing team processes. 
Conversely, SMM encompassing task content (e.g. shared knowledge related 
to task requirements) is directly related to performance. Future research 
might distinguish relationships between measurement methods. Specifically, 
systematically comparing between different knowledge elicitation methods 


(Smith-Jentsch, 2009) for SMM (e.g. concept mapping) and determining how 


that moderates the relationship between SMM and performance would be one 
fruitful direction for future research. Additionally, investigating the 
moderating effect of similarity versus accuracy of the team’s SMM would be 
a substantial contribution to the literature. 

Team composition also has a number of different definitions and 


conceptualizations which makes interpretation of the literature difficult. 


However, we suggest referencing two aspects of team composition: 
configural operationalization and variable type (e.g. conscientiousness). Each 
of these aspects has been shown to have an effect on outcomes of interest 
(e.g. Bell, 2007; Tekleab & Quigley, 2014). A salient direction for future 


research is determining how configural operationalizations differentially 


influence the relationship between various team composition variables and 
team outcomes. Similarly, the term “team virtuality” should be viewed as a 
high-level construct encompassing many variations in terms of measurement 
and operationalizations. 

We suggest that some of the relevant aspects of virtuality for 
categorization purposes include geographic distribution, reliance on virtual 
tools, characteristics of the tools being utilized (e.g. synchronicity and 


informational richness, Kirkman and Mathieu, 2005), and relative amount of 


face-to-face and electronic media usage (Kirkman, Rosen, Tesluk, & Gibson, 
2004). As most studies mix and ultimately confound several of these 
characteristics when measuring team virtuality, it may be difficult to 
generalize the results of most studies. However, these characteristics provide 
an initial foundation by which to understand how virtuality is measured and 
how it might impact team results. Yet another future research direction is to 
disentangle the relationships among these various aspects of virtuality and 
determine how they influence various team  process—performance 
relationships. 

Finally, the team adaptation literature has also been inconsistent due to a 
number of different research approaches to defining and studying this 
construct. However, Maynard et al. (2015) synthesized the relevant literature 
and broadly categorized this research as having three focal approaches: 


conceptualizing team adaptation as a process, team adaptation as an outcome, 


and team adaptability or the team’s inherent ability to adapt. Specifically, 
Maynard et al. (2015) distinguished between team adaptation as a process 
and team adaptation as an outcome by specifying that: “team adaptive 
outcomes are the consequences of the adaptation process, which may include 
constructs such as: various emergent states such as team cognition, team 
members’ affective reactions such as willingness to work together again, 
team effectiveness, and team performance” (p. 654). Taken as a whole, team 
adaptation and the other four teamwork constructs discussed in this chapter 
are associated with varying approaches to  conceptualization and 
measurement; future researchers should be careful to rectify these 


inconsistencies in appropriate and meaningful ways. 


Fostering Expert Team Elements 


Various elements that contribute to an expert team’s success, and 
differentiate an expert team from others, were identified within this chapter. 
The team aspects necessary for ensuring high performance (despite dynamic, 
uncertain conditions and complex tasks), as well as characteristics that impact 
team functioning, were highlighted and discussed. By focusing on fostering 
high levels of these components, effective team performance can be achieved. 
There are steps organizations can take to facilitate these integral teamwork 
elements. Specifically, we provide several recommendations for 
organizations and practitioners to follow to foster higher levels of these 
teamwork elements, and potentially transform a team of experts into an 
expert team. We note that selection may be helpful in ensuring an expert 
team’s success but, due to practical constraints, it is important to focus on 
what can be undertaken after a team has already been assembled to improve 
success. 

One of the most important ways in which teamwork can be facilitated 


and improved, including within the context of expert teams, is through the 


provision of team training (Burke et al., 2004; Hughes et al., 2016). There is 
ample empirical evidence supporting the use of team training to foster 
improved levels of knowledge, skills, abilities, and other characteristics 


necessary for higher performance (e.g. Salas et al., 2008a). The effectiveness 


of team training can be enhanced in multiple ways (e.g. conduct a needs 
analysis to inform training content), and these methods are also purported to 
be effective for improving training outcomes with expert teams (e.g. Burke et 


al., 2004). However, this is beyond the scope of this chapter; how to enhance 


team training effectiveness has been thoroughly reviewed elsewhere (e.g. 
Burke et al., 2004; Gregory, Feitosa, Driskell, Salas, & Vessey, 2013; Salas 
et al., 2009a). Rather, we highlight the importance of considering the 
implementation of team training in conjunction with promoting a supportive 
organizational climate. 

Organizational climate can be defined as employees’ perceptions of 


organizational policies, procedures, and practices (Denison, 1990; Ostroff, 


Kinicki, & Tamkins, 2003). Cultivating an organizational climate that 
supports the use of learned behaviors after training has ceased is a critical 
step in ensuring that trained behaviors are used on the job (Baldwin & Ford, 
1988). We also suggest that, in the absence of training, ensuring that different 
aspects of the organization are in alignment with the goal of cultivating an 
expert team, rather than through training or other methods, is critical. This 
goal can be achieved through several strategies. In particular, clearly 
delineating team goals can be helpful (Cannon-Bowers, Salas, & Milham, 
2000), such as outlining the types of behaviors (e.g. adaptive performance) 
that organizations want teams to exhibit. Employees should also be afforded 
numerous opportunities to practice and use the skills that are required for 
effective performance as well as have access to any equipment required for 
doing so (Ford, Quifiones, Sego, & Sorra, 1992; Lim, 2000, 2001). Another 
approach that has been demonstrated to increase the implementation and use 
of desired behaviors is support from supervisors and co-workers (Ford et al., 
1992; Russ-Eft, 2002; Smith-Jentsch, Salas, & Brannick, 2001b). Scheduling 


meetings with employees and emphasizing the necessity of encouraging and 


promoting certain behaviors, even among co-workers, may be helpful in this 
respect. 


Yet another way in which the organizational climate can be modified to 


encourage expert team performance is through the implementation of team- 
level rewards for demonstrating behaviors that denote expert teamwork 
(Cannon-Bowers et al., 2000). We also suggest that team members should be 
encouraged to openly discuss performance and mistakes, with a focus on how 
team-based behavior can be improved in the future. This might entail 
scheduling debriefing sessions after performance episodes. Organizations 
should also set aside resources to evaluate team activity over time, measuring 
both team performance and the elements necessary for improving it, such as 
shared cognition. This can enable organizations to determine if teams are 
improving as a result of steps that have been enacted to facilitate increased 
team performance. There are numerous sources describing methods for 
improving measurement which we suggest should be referenced before 
devising a measurement strategy (e.g. Baker & Salas, 1992; Rosen et al., 
2008). 


Conclusion 


Team research has come a long way in ten years; both theory and practice 
have advanced. Expert teams are able to overcome challenges, effortlessly 
perform within strenuous environments, and solve complex problems. 
Successful expert teams engage in processes that emerge in a compilational 
manner, thereby becoming a team that is more efficacious beyond just the 
sum of each individual’s level of expertise. Specifically, these teams have 
high levels of team adaptability and shared cognition; they engage in team 
leadership and are able to overcome threats due to suboptimal team 
composition and virtuality. Although all teams of experts are composed of 
highly experienced and skilled individuals, not all are worthy of the title of 
“dream team.” The critical competencies outlined in this chapter provide a 
roadmap and foundation for highly skilled teams to approach “dream team” 
status. 

Ultimately, organizations have many opportunities and methods through 
which expert team performance can be encouraged and improved. This 
chapter delineates the behavior and teamwork elements that are necessary to 
transform a team into an expert team and discusses how these various aspects 
can be fostered. Specifically, ensuring that organizational practices, polices, 
and procedures are in alignment with desired team attitudes, behaviors, and 
cognitions is critical. As previously noted, selection may be one method 
through which expert team success can be cultivated but evidence suggests 
that organizational climate also has an influence on a team’s level of 


performance (e.g. Smith-Jentsch et al., 20016). In carefully reviewing and 


implementing the suggestions outlined here, organizations may facilitate an 


organizational climate supportive of expert team success. 
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Introduction 


Individual differences in musical achievement have at all times awed 
musicians and audiences alike. In former times, royalty and nobility invited 
outstanding musicians to perform in their salons. Today, the general public 
crowds the concert halls when certain celebrities perform while other 
concerts are scarcely attended. Mass media and social media pay special 
attention to child prodigies or musical savants. It is therefore interesting to 
ask why some performers achieve outstanding levels of performance while 
others do not, and we may further ask how certain types of practice may 
contribute to different ultimate developments. This chapter surveys research 
on individual differences in the music domain, emphasizing the importance 
of quantity and quality of practice. The first two main sections situate this 
research in the larger scholarly context. The concluding section opens the 
horizon regarding future studies and reconnects the findings to the societal 
context. 

Scientific attempts to understand individual differences have been 
undertaken for a long time. In the late 1700s Lord Barrington investigated 
Mozart’s early performance achievements, and other case studies have 
followed, particularly in Carl Stumpf’s circle in Berlin in the beginning of the 
twentieth century (Kopiez & Lehmann, 2016). Although such single case 
studies are highly informative, they usually do not suffice for modern 
scientific standards. They merely document high achievements under 
controlled conditions, and even though Stumpf began to develop systematic 


psychometric testing, often researchers explain exceptional performance in 


terms of innate musical capacities. While Billroth’s (1895) publication Who 
is musical? can be seen as a Starting point for research on musical abilities in 
the nineteenth century, it was the 1920s and 1930s that witnessed the 
development of “measures of musical talents,” which assessed subjects’ 
perceptual discrimination abilities (Seashore, 1938). With few exceptions, 
most music aptitude tests have tried to predict the potential for music 
performance (for a review, see Boyle, 1992). Their overall success was 
limited, however, probably because the effects of predispositions tend to be 
confounded with the amount of previous training, which is rarely statistically 
controlled for. Generally, individuals of higher musical proficiency tend to 
score higher in such ability tests (Law & Zentner, 2012). Ability testing 
emphasizes internal stable dispositions. 

Recent research has tried to tease out the natural abilities that play a part 
in the development of any form of expertise including music. Because music 
educators are often confronted with folk-psychological assumptions of 
parents regarding their child’s talent or lack thereof regarding music abilities, 
the scientific debate about the heritability of genetic influences on music 
giftedness, talent, and practice is highly relevant in the domain of music. 
Expertise research has been hotly discussed by nature theorists, who have 
themselves tried to put together evidence for the influence of genetic 
influences (Mosing, Pedersen, Madison, & Ullén, 2014; Pulli et al., 2008; 
Tan, McPherson, Peretz, Berkovic, & Wilson, 2014). However, how research 
on novice populations may transfer to expert performance is not clear 
(Ericsson, 2014; Ericsson, Roring, & Nandagopal, 2007). Since early 
exceptional achievements are of particular relevance, i.e. musical prodigies, 
the curious reader should consult pertinent chapters in McPherson (2016). 


Current middle-ground theories promote complex models in which genetics 


and the environment interact. One promising approach is also to consider 
self-regulation and the motivation (need) to practice, which may result in 
large amounts of relevant training (Evans & Bonneville-Roussy, 2015). 

Biographical accounts of child prodigies and other performers in history 
and the present attest to the high level of investment on the part of the social 
environment (e.g. Csikszentmihalyi, Rathunde, & Whalen, 1993; Kopiez & 
Lehmann, 2016; Lehmann & Kristensen, 2014; McPherson, Davidson, & 
Faulkner, 2012). For instance, the socio-economic conditions of a young 
musician’s family constitute obvious factors that influence the choice of a 
teacher, the quality of the instrument played, and other possibilities afforded 
to the learner. An analysis of successful Polish musicians by Manturzewska 
(1995) revealed a common pattern of attitudes, value systems, and family 
structure in the musicians’ families of origin. Families were emotionally 
stable, task oriented, and careful in selecting their children’s friends, and they 
strongly supported the musical activities. These attitudes gained importance 
as they translated into behavioral consequences in the daily lives of 
musicians. For example, Csikszentmihalyi et al. (1993) demonstrated that 
families of high-achieving children changed their lives to accommodate the 
needs of their talented offspring by exempting them from household chores to 
give them additional time to practice. 

Although practice is omnipresent during the development of expertise in 
music, its role and manifestation are not identical in all musical genres. 
Different musical genres (classical, folk, popular) are characteristic of 
specific musical cultures, and those different cultures have their respective 
types of learning practices and activities, motivational settings, and 
competencies demanded (Bézenac & Swindells, 2009). Thus far, most 


research on musical expertise has been conducted in the classical 


conservatoire tradition — also known as the “Western art music tradition.” 
Investigating expert performance in varying genres may yield differing 
results. For example, whereas an early start of training is typical for pianists 
and violinists in the classical music domain, jazz guitarists start much later 
(Gruber, Degner, & Lehmann, 2004), and so do most of today’s classical 
singers (Kopiez, 1998); while solitary practice is prevalent for classical music 
soloists, jazz soloists and vernacular (folk) musicians also require extensive 
practice in groups (“jamming”). However, important commonalities should 


exist regardless of the specific music style in question. 


Increasing Performance through Practice 


Practice: Investing the Time 


A discussion about the role of training-induced changes in performance was 
triggered by Ericsson, Krampe, and Tesch-Rémer (1993), who first 
introduced the concept of “deliberate practice.” Deliberate practice is a set of 
structured activities that experts in the domain consider important for 
improving performance; it is often strenuous and can therefore only be 
maintained for limited amounts of time per day without danger of 
psychological or physiological burnout. Although it may be difficult to refute 
claims by famous musicians not to have practiced much or to hate practice, 
the empirical evidence regarding contemporary musicians makes such claims 
rather suspicious (Chaffin, Imreh, & Crawford, 2002). Musicians are likely to 
engage in conscious impression management when belittling practice in the 
classical tradition or, in the case of rock and popular musicians, to dismiss the 
role of formal instruction by emphasizing self-teaching, that is, autodidactic 
learning (Green, 2002). 

At the Berlin University of Music, Ericsson et al. (1993) investigated 
violin students from different degree programs that varied with regard to the 
instrumental proficiency required. The students were interviewed 
retrospectively about their practice and skill development. The amount of 
lifetime accumulated practice up to the point of the interview (or even the 
point of entry into the academy) was clearly related to the level of 
performance attained. Less-proficient performers had practiced less than 
more-skilled ones. The lifetime trajectory of practice reported by the most 
promising group of students resembled that of musicians currently employed 
in Berlin orchestras. The results underline the predictive validity of the 


accumulated hours as an indicator of excellence, and are hence at odds with 


the everyday belief that some musicians — the “highly talented” ones — need 
not practice as much as the less talented, who have to compensate for lack of 
talent with excessive practice. 

Sloboda, Davidson, Howe, and Moore (19965) replicated Ericsson et al.’s 
(1993) study with sub-experts: students aged 8 to 18 years from a specialist 
music school and children of various levels of musical performance; and 
children who had ceased to play their instruments. The students were 
interviewed by the researchers about their practice history. In addition, many 
other types of data were collected, including a 42-week longitudinal 
recording of practice diaries. The results clearly support the assumptions 
obtained by Ericsson et al. (1993). The least proficient group of students had 
practiced less than the better-performing groups, and the students who 
dropped out of music lessons had practiced even less. In order to proceed 
from one level of performance to the next in a standardized syllabus system, 
the best groups’ increase in practice was even larger than expected. Thus, 
already relatively early in instrumental music learning, the amount of practice 
is significantly related to level of performance. These results have been 
successfully replicated in different domains. Note that mere durations are 
only a proxy to the amount of deliberate practice invested, since 
retrospectively estimated practice durations encompass bouts of highly 
effective practice as well as stretches of suboptimal practice (see discussion 
in Platz, Kopiez, Lehmann, & Wolf, 2014). 

Some researchers have wondered about the increased practice times of 
higher achieving students. They propose that talented children could practice 
more in a “rage to master” the skill (Winner, 1996) or that the tangible 
progress and initial success may keep some children motivated to practice, 


acting as self-reinforcement. This success would also result in heightened 


self-regulatory skills and self-efficacy beliefs (McPherson et al., 2012). In 
essence, the reasons behind intensified practice remain somewhat unclear, but 
success has been achieved in linking practice durations to underlying motives 
(Evans, McPherson, & Davidson, 2012). Furthermore, long-term 
commitment seems to play a role in predicting involvement and practice. 
Evans and McPherson (2015) did a ten-year follow-up study and found that 
commitment to playing an instrument influenced performance after three 
years and even after ten years. 

It is noteworthy that the number of hours necessary for achieving 
particular levels of performance is not a constant across all musical 
instruments. Jorgensen (1997) showed that different instrumental groups 
practice very different numbers of hours. Pianists and violinists tend to be 
practice fanatics, logging the most hours, followed by other strings, organ, 
woodwinds, and brass, closing with the singers at the bottom of the list. Such 
differences may result from different demands that instrumental performance 
imposes on the body. In the case of singers, different educational traditions 
may have an influence (Kopiez, 1998). Although duration of practice is 
predictive of long-term success, it might not be as indicative of performance 
in the short run, for example, when learning a specific piece of music 
(Williamon & Valentine, 2000). A player’s prior knowledge of music might 
influence actual practice times. For example, those who have not worked 
systematically on music by Bach may face problems that experienced 
Baroque performers do not (Lehmann & Ericsson, 1998a). 

The concept of deliberate practice denotes different activities for 
different musical styles and sub-skills. For musicians playing classical 
repertoire, a large portion of practice is solitary practice, working on 


instrumental technique and acquiring new pieces, assisted by more or less 


regular visits to a teacher. For jazz musicians, in addition to solitary practice, 
a substantial part of practice is communal practice with other musicians. 
Sitting in jam sessions, listening to others playing, and copying performances 


by famous musicians from recordings all constitute activities that improve 


performance (Gruber et al., 2004; Nielsen, 2015). For singers and some 
instrumentalists, working with an accompanist is an important practice 
activity, and orchestra conductors first familiarize themselves with a piece by 
silently reading and imaging the score with the desired interpretation before 
working with the ensemble. Thus, deliberate practice is goal-directed, 
optimized practice that is directly related to typical demands of a task 
imposed by the domain. Disregarding the necessary match between task- 
relevant activity and performance demand leads to a weakening of the 
predictive power of practice durations (as in Hambrick et al., 2014). Using 
rigorous matching criteria, a meta-analysis by Platz and colleagues (Platz, 
Kopiez, Lehmann, & Wolf, 2014) reported a cumulative corrected correlation 
of 0.61 between task-relevant training and performance. What other factors 


might play a role will have to be assessed in future studies. 


Beyond Time: Investing the Effort 


Ericsson et al. (1993) stressed that practice could lack inherent enjoyment 
because it requires much mental and physical effort. Musicians may enjoy 
their own improvement but dislike the actual practice activity. In a survey 
study on practicing, musicians indicated that performing in front of an 


audience was most enjoyable but highly effortful (Lehmann, 2002), whereas 


learning new pieces and working on difficult spots was most effortful and not 
enjoyable. Apparently, activities that resemble the target activity of 
performing seem more enjoyable than activities of a preparatory nature, even 
though the latter’s relevance for improving skills is unquestioned. Laboratory 
studies by Hyllegard and Bories (e.g. 2009) support these findings. Recent 
music education research showed that more proficient students were able to 
overcome initial lack of motivation through volitional control (Roth, 2012). 
From this it follows that enhancing the quality of practice requires substantial 
efforts and metacognitive skills (Williamon, 2004). 

Since effort and learning are not directly observable (e.g. mere duration 
or use of certain strategies) they have to be judged by the co-occurrence of 
certain cognitive processes (Chaffin & Imreh, 2001). Many researchers in the 
field use verbal report methodologies to get at these difficult-to-observe 
processes. For example, Chaffin et al. (2002) published an extensive case 
study about the solitary practice of a classical musician. In a naturalistic 
setting, they followed a performer practicing a new piece for performance 
and obtained retrospective and concurrent reports. The authors found 


systematic activity with predictable stages and activities which all serve to 


establish a strong internal representation of the piece and the conditions under 


which the performance will take place. 


1. In the first phase, the musician tries to get the “big picture” of the 
piece. The first stage entails reading through the piece or more generally 
getting an aural representation of the entire piece. Practice strategies 
vary substantially according to the working habits of the musician; 


sight-reading, analyzing, or listening to recordings are preferred. 


2. In the second stage, technical practice is undertaken to master the 
piece. During the second stage the piece is worked on in sections, which 
increase in length as practice progresses. Whether the artistic 
interpretation is developed during the course of learning to play the 


piece or during anticipatory analytic processes might be a matter of 


individual preferences and habits (Hallam, 1995). During this 
elaborative stage of practice, the motor programs become largely 


automatic and the piece is being memorized. 


3. Next, in the third stage the actual stage performance is tried out. 
During this stage, performance is prepared more directly by putting the 
pieces together and ironing out the seams between them. Memory, 
which up to then was more implicit, is now deliberately assisted by 
creating an internal map of the piece, knowing the order of the parts as 
well as points where the performer could restart in case of a memory 
lapse during performance. During this stage the piece is polished by 
slow playing, playing for an imagined or an informal real audience, 
refining interpretation details, and bringing all sections up to or even 


slightly above the correct tempo. If possible, the musician even practices 


under ecologically valid performance conditions (in concert attire, in 


different locations). 


4. Since such final polishing and preparation work has diminishing 
returns, further practice is considered to be maintenance work. This 
constitutes the fourth stage, which sometimes extends over very long 


periods of time. 


Practicing is an effortful activity and a skill that has to be learned. Gruson 
(1988) demonstrated that experts differed from novices in their practice skill. 
A number of studies revealed that (adult) supervision during practice is 
important for beginning musicians (Davidson, Howe, Moore, & Sloboda, 
1996; Lehmann, 1997). In the simplest case, the adult or coach ensures that 
time is spent with the instrument. Preferably, goals and feedback are 
provided. Research suggests that not all parents or tutors necessarily have to 
be musicians — everyone can hear wrong notes, encourage lovingly, or simply 
watch the clock. However, mothers’ previous experience with leaming a 
musical instrument may influence their ideas about how much practice is 
necessary and their ability to support their child’s practice (McPherson & 
Davidson, 2002). Renwick and McPherson (2002; also McPherson et al., 
2012) showed that children practicing by themselves engaged in elaborated 


activities when they were motivated by the piece, but simply played through 
the piece when uninterested in it. Similarly, when the goal is to master a 
certain piece or a specific difficulty, adults may work hard and use more 
practice strategies than when they want to enjoy themselves or relax rather 
than mastering the instrument (Lehmann & Papousek, 2003). After the 
musicians have developed metacognitive skills, they can take over to regulate 


their practice themselves — if they are motivated to do so. Research on self- 


regulation has received much interest in recent years (Araujo, 2016; 
McPherson et al., 2012; McPherson & Zimmerman, 2002). 


The Development of Musical Expertise 


Stages and Phases 


Demarcating points along the time line of acquisition of a new skill allows us 
to better conceptualize the process as a whole, and several stage and phase 
models have been proposed. Fitts and Posner (1967), for example, in their 
well-known model describe skills as being first cognitive, then associative, 
and later autonomous, in essence requiring less and less cognitive mediation 
as skilled performance increases. Similar stages were identified by Sudnow 
(1993) in his phenomenological account of his learning to improvise jazz on 
the piano. First, he had to decide consciously which chord to use next and 
then how to distribute the chordal notes on the keyboard (voicing). Later his 
fingers seemed to find the right notes by themselves. Much later, his aesthetic 
decision of what to play seemed to trigger the correct chord sequences with 
associated voicings. Whereas the focus of attention in novices is directed 
toward technical, low-level aspects, experts attend to higher level, strategic or 
aesthetic issues, a finding also demonstrated in improvisation (Hargreaves, 
Cork, & Setton, 1991). For most professional musicians, the earlier stages of 
skill development are successfully completed in (early) childhood. Fitts and 
Posner’s (1967) model is informative with regard to skill development of an 
individual, but it neglects the lifespan context. 

Bloom (1985) explicated how skills develop through life. First, the child 
is introduced to the domain in an informal phase, and it is here that children 
in “musical” households may be at an advantage. Then comes a phase during 
which formal tuition is sought. This stage extends until the young musician 
makes a full-time commitment to music in order to become a professional (or 
decides to remain an amateur or drop out of music making altogether). 


McPherson et al. (2012) adapted a model from sport psychology with a 


“setting there” phase in three stages and a “staying there” stage with 
maintenance as the focus. Once a professional status has been reached, the 
expert is working at trying to make lasting contributions (e.g. making sound 
recordings for major record labels, playing in prestigious concert halls, 
becoming an acclaimed teacher later in their career). The psychomotor 
adaptations do not decline inevitably with old age but can be maintained for a 
long time through continuous practice through the well-known mechanism of 
selective optimization with compensation. Jabusch and colleagues (Jabusch, 
Alpers, Kopiez, Vauth, & Altenmiiller, 2009) were able to predict regularity 
in scale playing in adult pianists from the practice time accrued between 
measurements (two years). Krampe and Ericsson (1996) demonstrated that 
older pianists were able to counteract losses in motor performance through 
practice, whereas non-pianists did not show this advantage (see Krampe & 
Charness, Chapter 42, this volume). However, self-concept and self- 
estimated performance ability suffer nevertheless (Gembris & Heye, 2014). 
The time needed for experts to develop sufficient skills for a 
professional career is sometimes estimated to be at least a decade (Ericsson & 
Crutcher, 1990). Hayes (1989) demonstrated that this “ten-year rule” also 
applied to composers in classical music, including Mozart. Works from 
Mozart’s earliest phases were conspicuously underrepresented in selected 
lists of his recordings. Similarly, Weisberg (1999) demonstrated that it took 
The Beatles approximately a decade to acquire an international reputation. 
Prior to writing their own songs they covered music by other bands. While it 
is futile to dwell on the exact number of years or hours as they may vary from 
domain to domain, it is important to note that even famous exponents of a 
domain take a long time to acquire their skills. How much the typical 


development can be accelerated in prodigies is to date unknown. In order to 


compete successfully for scholarships, prizes, and media attention, 
instrumentalists in the classical music domain have to master the most 
demanding repertoire as teenagers. This requires either an early start for 
highly competitive instruments (e.g. violin, piano) or the possibility of 
transfer of knowledge and skills from previously played instruments to those 
instruments that do not allow such an early start (e.g. string bass, oboe, 
trombone). Altogether, the development of expert performance can be seen as 
an adaptation to the typical task constraints of the domain (Ericsson & 
Lehmann, 1996), involving changes in cognitive, physiological, and 


perceptual-motor parameters that facilitate superior performance. 


Cognitive Adaptations 


Among the cognitive adaptations in music are aspects of problem-solving and 
memory (Snyder, 2016). The former can be seen in virtually all domains of 
expertise (Ericsson & Lehmann, 1996). Even when memorization is not their 
explicit goal, experts tend to have excellent long-term retention for domain- 
related material. For example, incidental memory for music just played 
correlated moderately with accompanying ability in classical pianists 


(Lehmann & Ericsson, 1996). Musicians recall musical material better than 


non-musicians, especially when the material is structured according to rules 
of tonality (Snyder, 2016). Expert—novice differences decreased when 
tonality rules were violated or when random note sequences had to be 
recalled. This skill-by-structure interaction, demonstrated also in other 
domains, documents that experts’ advantages are largely due to their 
knowledge and how their memory skills have adapted to the structure of the 
stimuli. Most likely, some of the gains can be attributed to a development of 
working memory (Nutley, Darki, & Klingberg, 2014). 

Acquired domain knowledge has been the most prominent explanation 
for the superiority of expert performance. Scientists across many different 
domains showed that the essential factor for development of expertise is the 
accumulation of increasingly complex patterns in memory. It has been shown 
that expert knowledge can be retrieved quickly using long-term working 
memory (Ericsson & Kintsch, 1995). Over the years Chaffin and 
collaborators have shown convincingly how musicians developed such 
sophisticated mental representations with associated retrieval structures that 


lead to successful performance of the rehearsed piece from memory even 


under high-stress conditions on stage (Chaffin, Demos, & Logan, 2016). The 
study of individuals’ cognitive representation of musical structure is 
important for understanding how music performance works, because it helps 
to understand why certain mistakes happen and in which way a good use of 
the knowledge can be supported (Palmer, 1997). But also from an educational 
point of view it is relevant to know how different learning processes or 
methods may result in different representations (Gruhn & Rauscher, 2002). 

A particularly impressive effect of the impact of knowledge for musical 
performance was found in studies with autistic savants. Despite the cognitive 
and communicative limitations that prevent them from functioning normally 
in everyday contexts, some autistic savants have exceptional musical skills 
and can play back music after only a few hearings (Ockelford, 2007). It can 
be demonstrated that these skills are based on knowledge-related generative 
processes (Sloboda, Hermelin, & O’Connor, 1985). When confronted with 
atonal music, the savants fail to imitate the music but simply play 
haphazardly, suggesting that familiarity with the material and the genre 
mediates memory performance (Charness, Clifton, & MacDonald, 1988). The 
phenomenon of savants’ music memory demonstrates that making use of 
one’s knowledge about the structure of the stimulus is a quick and automatic 
process. 

Musical knowledge comprises not only knowledge about musical pieces, 
but the cognitive mechanisms to represent and manipulate the relevant 
knowledge. Lehmann and Ericsson (1997) suggested a triangular model of 
mental representations for musicians. In brief, musicians first need to imagine 
their anticipated, desired outcome. Next, they have to represent their current 
ongoing performance in order to compare it to their original plans. Finally, a 


mental representation is necessary of how a particular plan can be 


implemented on the instrument — how it feels (see Woody, 2003, concerning 


motor production representations). Woody (1999) investigated the connection 
between ongoing and desired performance. Pianists were asked to imitate the 
artistic, expressive features of a model musical performance; verbal reports 
were recorded to ascertain which features they explicitly identified. 
Performance data showed that they imitated more accurately those features 
that they had correctly identified in their verbal reports. In a study on African 
master drummers, researchers found that rhythms in triple meter such as the 
Bolero rhythm were very difficult to imitate, because they tried to assimilate 
the rhythms to African rhythmic prototypes (Kopiez, Langner, & Steinhagen, 
1999). Thus, even seemingly automatic performance is mediated by complex 
cognition, even at high levels of proficiency. 

In addition to knowledge-related cognitive adaptations, we can observe 
changes in the use of sophisticated metacognitive and self-regulation skills in 
musical learners. Gruber, Weber, and Ziegler (1996) analyzed top-level 
orchestra musicians and above average amateur musicians. Judging 
retrospectively, experts indicated higher levels of aspiration along with a 
more positive attitude toward performance situations compared to the 
amateurs. As regards their current situation, experts rated themselves more 
effective in their learning behavior, but did not differ from the amateur 
players concerning their motivation. Gruber et al. (1996) surprisingly found 
that experts’ competence and control beliefs were weaker concerning musical 
performance than regarding everyday life. This leads to an ambiguous 
situation: on the one side, early on experts practice effectively, look for 
challenging performance situations, and have aspirations; on the other side, 
once they work within a community of experts, where they are only one 


among many, they neither perceive themselves as outstanding nor do they 


have superior self-concepts. The interplay of self-regulation, self-efficacy, 


and self-concepts probably differs between experts and non-experts. 


Physiological Adaptations 


Everyone knows the minor physiological adaptations that happen in response 
to habitual usage of our bodies in everyday life. These adaptations are 
specifically localized, such as the growth of muscle after a few days of bike 
riding or the emergence of calluses on fingertips after starting to play the 
guitar or working in the yard. Musicians undergo a number of less obvious 
but highly telling adaptations. For example, Wagner (1988) found that degree 
of forearm rotation differed systematically between pianists (larger extent of 
inward rotation), violinists (larger outward rotation), and controls. However, 
the overall degree of rotation remained constant in all three groups but was 
shifted toward the respective habitual usages for the instrumentalists. Singers 
and brass players were found to have significantly larger vital and total lung 
Capacities compared to controls (Sundberg, 1987). And the superior 
inhalation and expiration pressures in trumpet players were found only after 


several long notes were played (Fiz et al., 1993), demonstrating the highly 


contextual specificity of such changes. 

Additional links between training and certain adaptations were 
uncovered in recent efforts to understand how the brain processes music, 
especially through the use of imaging techniques (for a review, see Miinte, 
Altenmiiller, & Jancke, 2002). The first study that received widespread 
attention was one that found that the cortical representation of the fingers of 
the left hand in string players was enlarged compared to that of the thumb 


(Elbert et al., 1998). No changes occurred with the representations of the 


fingers of the right hand (the bowing arm) and this cortical reorganization 


was more pronounced for subjects who had started musical training at an 


earlier age. Research in neuroscience has since been very interested in music 
training (for a review, see Herholz & Zatorre, 2012). 

Further studies, especially those comparing experts with novices, 
showed that cortical reorganization was not restricted to playing music but 
also occurred when listening to music or speech. Pantev and colleagues 
(Pantev, Roberts, Schulz, Engelien, & Ross, 2001) learned that larger areas of 
the cortex were activated involuntarily when musicians listened to tones of 
instruments they played. Or, differences in the volume of gray matter in the 
motor as well as auditory and visuospatial brain regions were found when 
comparing professional musicians (keyboard players) to amateur musicians 


and non-musicians (Gaser & Schlaug, 2003). Even left- (string) or right-hand 


(piano) dominant adaptations could be identified by coarse visual inspection 
compared to non-musicians (Bangert & Schlaug, 2006). We can safely 
assume that music training and practice lead to substantial functional and 
structural changes in a person’s brain and consequently, alters processing and 


performance capabilities. 


Perceptual-Motor Adaptation 


Instrumentalists require perceptual and motor skills different from those of 
non-musicians. For example, trilling on the piano requires ten to fourteen 
movements per second; jazz musicians have to respond to co-players within 
split seconds. Motor researchers found that pianists were able to tap faster 
and more accurately than controls with their fingers, but that this advantage 
did not transfer to their heels (Keele, Pokorny, Corcos, & Ivry, 1985). How 
information is acquired with the senses also changes qualitatively. For 
example, there are instrument-specific pitch perception preferences, whereby 
some musicians are either more oriented toward the fundamental pitch (often 
players of lower-pitch melodic instruments) while others listen for spectral 
aspects (often players of percussive or high-pitched instruments; Schneider, 
Sluming, Roberts, Bleeck, & Rupp, 2005). These preferences are correlated 
with asymmetries of the brain structure. A meta-analysis by Mishra (2013) 
uncovered that music sight-reading did improve with overall musical ability 
and was not simply a highly trained visuomotor decoding process (see also 


Lehmann & Ericsson, 1996). Finally, musicians develop a finer frequency 


and loudness discrimination than non-musician controls (Houtsma, Durlach, 


& Horowitz, 1987). Musicians playing instruments that require fine tuning of 


individual notes during performance develop a more accurate discrimination 
for pitch height, whereas percussionists, whose work relies heavily on 
discriminating rhythms, show an improved perception of auditory duration 
(Rauscher & Hinton, 2003). Likewise, pianists require increased sensitive 
tactile discrimination, which proved to be related to the amount of practice 


undertaken (Ragert, Schmidt, Altenmiiller, & Dinse, 2004). Pianists show co- 


activation of the auditory cortex in the absence of auditory feedback when 
moving their fingers (Bangert, Jiirgens, Hausler, & Altenmiiller, 2006). The 
increased acuity of the senses and their intimate interrelatedness with the 
motor system are restricted to the stimuli musicians typically encounter when 


playing their respective instruments. 


Outlook: Pushing the Limits 


In this chapter, we have explored how music performance changes through 
practice. The debate is still open (and might remain so indefinitely) about 
what “natural” limits of performance exist for experts, and whether and how 
such limits can be pushed. Physiological factors might limit performance of 
selected individuals — more so at the novice level — but a number of 
environmental, historical, and societal factors have been identified that are 
likely to influence the upper bounds of performance at a given time in a given 
place. 

It is interesting to study in the history of a domain how the demands 
imposed on musicians have changed over time (Lehmann & Ericsson, 
1998b). Everyone is aware of changed standards in sports, where records are 
kept about achievements that have to be matched and surpassed by following 
generations of athletes. In music, the constraints of performance are related to 
the development of instruments. When the piano was invented in 1700, there 
was no specific way of playing it, and a standard repertoire did not yet exist. 
Later refinements of the instrument and the instrumental technique led to 
more complex compositions. A number of pieces exist that were deemed 
unplayable at the time of their composition, including examples even from 
the twentieth century (e.g., “Etudes” for guitar by Villa-Lobos; 
“Hammerklavier” sonata for piano by Beethoven; “Etudes” for piano by 
Ligeti; “Caprices” for violin by Paganini). Nowadays many of these pieces 
are standard fare for adolescent performers. Such historical increases in levels 


of performance result from specialization, improved training and practice 


methods, and from the extrinsic rewards a society offers to those who try to 
make eminent contributions to the domain. 

A bitter taste regarding the limits of expertise arises from the fact that 
most professional musicians suffer from medical problems. Common 
problems are hearing losses from overexposure to noise during practice or 
performance, as well as muscular-skeletal or neurological problems (see 


Kenny & Ackermann, 2016, for review). The ranking of instruments in order 


of prevalence of symptoms corresponds roughly to the intensity of practice 
required to reach high levels of performance, with pianists, violinists, and 
guitarists at the top of the list. There may also exist an upper limit for 
attainable performance with regard to the neuroplastic changes (Butler & 
Rosenkranz, 2006). Focal dystonia, a condition that affects high-level 
musicians and robs them of voluntary motor control of highly trained 
movements can result from abnormal cortical functioning and degraded 
representation of motor functions (Altenmiiller & Miiller, 2013). It is obvious 
that research about the physiological limits of musical performance and about 
interventions to overcome the limits (or remedy existing problems) is still at 
its very beginning. 

Similar to the domain of sports, where some disciplines are popular in 
certain countries but not in others, or where some countries provide 
incentives to reach the highest levels of performance, music is affected by 
societal factors. China, for example, developed a highly competitive piano 
instruction system after the end of the Cultural Revolution; playing the piano 
is now a highly valued and competitive cultural practice with a strong 
potential for a career. It is estimated that 40 million Chinese are seriously 
playing the piano. The resulting large number of highly qualified foreign 


students from Eastern European and Far Eastern countries entering 


performance degree programs in music academies in Western countries does 
not surprise. Researchers found differences in motivation to practice and 
work ethic between the foreign and the Western students (Comeau, Huta, & 
Liu, 2015). Thus, the cultural environment and its value and reward systems 
promote the development or neglect of skills in a given culture. 

Research reported in this chapter predominantly deals with the Western 
music tradition. To date, studies in non-European music genres are rare but 
would be interesting for many reasons. For example, Indian musicians are 
likely to show interesting problem-solving strategies because they perform 
mainly improvised music — as do musicians in the Middle East. Or Balinese 
musicians, who learn by ear an extensive repertoire, would offer insights into 
memory processes that are not mediated by music notation — as would to a 
certain degree European vernacular musicians in rock, popular, jazz, and folk 
music. New subdomains have emerged that constitute touchstones for 
theories previously developed in the classical music domain. For example, 
one should ask what constitutes practice for a DJ or a rap musician. Research 
in music expertise needs to take a broader look at different genres. 

The unique combination of affective, perceptual, cognitive, and time- 
critical motor aspects in music making along with its high cultural value 
make music a prime candidate for the study of complex skills. Children are 
introduced to music very early on in their lives — earlier than to most other 
domains of expertise — at a time when their brains and bodies are malleable 
and training can be most effective. This might be a significant factor in 
explaining the enormous differences we observe between musical experts and 
novices. The universal nature of music as a grammar-based but non-semantic 
temporal phenomenon theoretically allows studies in all cultures and across 


time, adding to the appeal of music as a domain for expertise researchers. 


Finally, the potential connections to music education in and out of formal 
learning contexts make expertise research a fruitful area of research for those 
whose interests concern how to make instruction and training effective and 


sustainable. 
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Musicians with extensive training and playing experience provide an 
excellent model for studying plasticity of the human brain. The demands 
placed on the nervous system by music performance are very high and 
provide a uniquely rich multisensory and motor experience to the player. As 
confirmed by neuroimaging studies, playing music depends on a strong 
coupling of perception and action mediated by sensory, motor, and 
multimodal integration areas distributed throughout the brain. A pianist, for 
example, must draw on a whole set of complex skills, including translating 
visual analysis of musical notation into motor actions, coordinating 


multisensory information with bimanual motor activity, developing fine 


motor skills in both hands coupled with metric precision, and monitoring 
auditory feedback to fine-tune a performance as it progresses. 

In this chapter, we summarize research on the effects of musical training 
on brain function, brain connectivity, and brain structure. First, we address 
factors inducing and continuously driving brain plasticity in dedicated 
musicians, arguing that prolonged goal-directed practice, multisensory motor 
integration, high arousal, and emotional and social rewards contribute to 
these plasticity-induced brain adaptations. Subsequently, we briefly review 
the neuroanatomy and neurophysiology underpinning musical activities. We 
focus on the perception of sound, integration of sound and movement, and 
the physiology of motor planning and motor control. We then review the 
literature on functional changes in brain activation and brain connectivity 
along with the acquisition of musical skills, be they auditory or sensory- 
motor. In the following section, we focus on structural adaptions in the gray 
matter of the brain and in fiber-tract density associated with music learning. 
We critically discuss the findings that structural changes are mostly seen 
when starting musical training after the age of 7, whereas functional 
optimization is more effective before this age. We then address the 
phenomenon of de-expertise, reviewing studies which provide evidence that 
intensive music-making can induce dysfunctional changes which are 
accompanied by a degradation of skilled motor behavior, also termed 
“musician’s dystonia.” This condition, which is frequently highly disabling, 
mainly affects male classical musicians with a history of compulsive working 
behavior, anxiety disorder, or chronic pain. Functional and structural brain 
changes in these musicians are suggestive of deficient inhibition and excess 
excitation in the central nervous system, which leads to co-activation of 


antagonistic pairs of muscles during performance, reducing movement speed 


and quality. We conclude with a concise summary of the role of brain 
plasticity, metaplasticity, and maladaptive plasticity in the acquisition and 


loss of musicians’ expertise. 


Performing Music as a Driver of Brain 
Plasticity 


Performing music at a professional level is one of the most demanding and 
fascinating human experiences. Singing and playing an instrument involve 
the precise execution of very fast and, in many instances, extremely complex 
movements that must be structured and coordinated with continuous auditory, 
somatosensory, and visual feedback. Furthermore, they require retrieval of 
musical, motor, and multisensory information from both short-term and long- 
term memory and rely on continuous planning of an ongoing performance in 
working memory. The consequences of motor actions have to be anticipated, 
monitored, and adjusted almost in real time (Brown, Penhune, & Zatorre, 
2015). At the same time, music should be expressive, requiring the 
performance to be enriched with a complex set of innate and acculturated 
emotional gestures. 

Practice is required to develop all of these skills and to execute these 
complex tasks. Ericsson and colleagues (Ericsson, Krampe, & Tesch-R6mer, 
1993) undertook one of the most influential studies on practice, with students 
at the Berlin Academy of Music. They considered not only time invested in 
practice but also quality of practice, and proposed the concept of “deliberate 
practice” as a prerequisite for attaining excellence. Deliberate practice 
combines goal-oriented, structured, and effortful practicing with motivation, 
resources, and focused attention. Ericsson and colleagues argued that a major 
distinction between professional and amateur musicians, and generally 


between more successful and less successful learners, is the amount of 


deliberate practice undertaken during the many years required to develop 
instrumental skills to a high level (Ericsson & Lehmann, 1996). 
Extraordinarily skilled musicians therefore exert a great deal more effort and 
concentration during their practice than less skilled musicians, and are more 
likely to plan, imagine, monitor, and control their playing by focusing their 
attention on what they are practicing and how it can be improved. 
Furthermore, they can be eager to build up a network of supportive peers, 
frequently involving family and friends. 

The concept of deliberate practice has been refined since it became clear 
that not only the amount of deliberate practice, but also the point in life at 
which intense goal-directed practice begins are important variables. In the 
auditory domain, for example, critical periods — “windows of opportunity” — 
exist for the acquisition of so-called “absolute” or “perfect” pitch. Absolute 
pitch denotes the ability to name pitches without a reference pitch. It is 
mediated by auditory long-term memory and is strongly linked to intense 
early musical experience, usually before the age of 7 (Baharloo, Johnston, 
Service, Gitschier, & Freimer, 1998; Miyazaki, 1988; Sergeant, 1968). 


However, genetic predisposition may play a role since absolute pitch is more 


common in certain East Asian populations and may run in families 
(Baharloo, Service, Risch, Gitschier, & Freimer, 2000; Gregersen, Kowalsky, 
Kohn, & Marvin, 2001). In the sensory-motor domain, early practice before 
the age of 7 leads to optimized and more stable motor programs (Furuya, 
Klaus, Nitsche, Paulus, & Altenmiiller, 2014a) and to smaller yet more 
efficient neuronal networks, compared to practice commencing later in life 
(Vaquero et al., 2016). This means that for specific sensory-motor skills, such 
as fast and independent finger movements, sensitive periods exist during 


development and maturation of the central nervous system, comparable to 


those for auditory and somatosensory skills (Ragert, Schmidt, Altenmiiller, & 
Dinse, 2003). 

The issue of nature vs. nurture, or genetic predisposition vs. 
environmental influences and training in musical skills is complex, since the 
success of training is itself subject to genetic variability. General observation 
suggests that outcomes will not be identical for all individuals receiving the 
Same amount of training. Evidence supporting the contribution of pre- 
existing individual differences comes from a large Swedish twin study 
showing that the propensity to practice is partially heritable (Mosing, 
Madison, Pedersen, Kuja-Halkola, & Ullén, 2014). In a series of studies, 
Schellenberg and colleagues investigated the contribution of cognitive and 
personality variables to music training, showing that those who engage in 
music perform better on cognitive tasks, have better educated parents, and 
describe themselves as more “open to experience” on personality scales 
(Corrigall, Schellenberg, & Misura, 2013). Findings are also beginning to 
accumulate in the music performance domain, indicating that learning 
outcomes can be predicted in part based on pre-existing structural or 
functional brain features (Herholz, Coffey, Pantev, & Zatorre, 2016). A 
convincing example of dysfunctional genetic predisposition is the inability to 
acquire auditory skills in congenital amusia: a hereditary condition 
characterized by absent or highly deficient pitch perception (Gingras, 
Honing, Peretz, Trainor, & Fisher, 2015). In the sensory-motor domain, 
musician’s dystonia, the loss of motor control in skilled movements while 
playing an instrument, has a strong genetic background in about one-third of 
affected musicians (Schmidt et al., 2009). 

On the other hand, training is clearly necessary for musical expertise, 


with a large number of researchers reporting that the length of musical 


experience is strongly correlated with performance on a range of musical 
tasks, as well as with brain function and structure (Amunts et al., 1997; 
Bengtsson et al., 2005; Bermudez, Lerch, Evans, & Zatorre, 2008; Chen, 
Penhune, & Zatorre, 2008a; Oechslin, Imfeld, Loenneker, Meyer, & Jancke, 


2010). Predispositions and experience contribute to musical expertise, and the 
relative balance between the two factors may differ in specific aspects of the 
many different musical sub-skills. Furthermore, it seems that there exist early 
sensitive periods during which musical stimulation or training of sub-skills 
has to take place in order to establish fertile ground for growing extra- 
ordinary expertise later in life. This is best illustrated by the scaffold 
metaphor (Steele, Bailey, Zatorre, & Penhune, 2013). An early start to 
training develops the “scaffold” for building a “skyscraper-like” level of 
expertise later in life, whereas a late start of training allows for only moderate 
results even after long and intense training. Of course these scaffolds may 
differ from one domain to the next. For example, an outstanding virtuoso like 
the legendary pianist Lang Lang, known for his breathtaking finger dexterity, 
may require both highly relevant inherited traits and intense early sensory- 
motor training. Other musicians such as the late French singer Edith Piaf, 
known for her emotional expressivity but somewhat lacking in technique, 
may have started technical exercises late in life but have genetic and 
biographical conditions allowing her to build up emotional depth, a character 
trait we feel and value, despite the difficulty in operationalizing it for precise 
study. 

Performing music at a professional level relies on a range of sub-skills, 
which are represented in different, though overlapping brain networks. 
Auditory skills such as the abovementioned perfect pitch, sensitivity to 


timing variations (e.g. “groove”) and to micro-pitches (e.g. tuning of a 


violin), or auditory long-term memory (e.g. memorizing a 12-tone series), are 
mainly processed in the temporal lobes of both hemispheres with a right 
hemisphere bias (Zatorre, 2001). However, signs of auditory and musical 
expertise can already be detected in the ascending auditory pathway at the 
brainstem level (Skoe & Kraus, 2013). Sensory-motor skills, such as low 
two-point discrimination levels and high tactile sensitivity (e.g. left fifth 
finger in professional violinists), bimanual or quadrupedal coordination (e.g. 
for piano and organ playing), fast finger movements (e.g. right hand 
arpeggios on the classical guitar), or complex hand postures (e.g. left hand on 
the electric guitar), are represented in premotor, motor, and parietal cortical 
areas, and in subcortical brain structures such as the basal ganglia and the 
cerebellum (Altenmiiller & Furuya, 2016b). Emotional and performance 
skills are supported by individualized prefrontal and orbitofrontal cortical 
regions and in the limbic system. Self-monitoring, anticipation of the 
consequences of one’s actions, motivation, and focusing attention (all 
contributing to goal-directed “deliberate” practice), recruit a highly diverse 
network, including lateral prefrontal cortices, parietal cortices, limbic 
structures, and particularly motivational pathways, including the accumbens 
nucleus, and memory structures such as the hippocampus (Zatorre & 


Salimpoor, 2013). All of these regions and the interconnecting nerve fibers 


are subject to modifications in function and structure in association with 
musical practice, a phenomenon which is based on brain plasticity. 

Brain plasticity denotes the general ability of our central nervous system 
to adapt throughout the lifespan to changing environmental conditions, body 
biomechanics, and new tasks. Brain plasticity is most typically observed for 
complex tasks with high behavioral relevance and those which activate 


circuits involved in emotion, motivation, and reward. The continued activities 


of accomplished musicians are ideal for providing the prerequisites of brain 
plasticity (for a review, see Schlaug, 2015). In musical expertise, the 
abovementioned processes are accompanied by changes in the function of the 
brain’s neuronal networks, as a result of a strengthening of synaptic 
connections, and in changes of its gross structure. With respect to 
mechanisms and microstructural effects of plasticity, our understanding of 
the molecular and cellular processes underlying these adaptations is far from 
complete. Brain plasticity may occur on different time scales. For example, 
the efficiency and size of synapses may be modified in a time window of 
seconds to minutes, while the growth of new synapses and dendrites may 
require hours to days. An increase in gray matter density, which mainly 
reflects an enlargement of neurons due to increased metabolism, needs at 
least several weeks. White matter density also increases as a consequence of 
musical training. This effect is primarily due to an enlargement of myelin 
cells which wrap around the nerve fibers (axons) and dendrites, greatly 
contributing to the velocity of the electrical impulses traveling along them. 
Under conditions requiring rapid information transfer and high temporal 
precision these myelin cells adapt by growing, and as a consequence nerve 
conduction velocity increases. Finally, brain regions involved in specific 
tasks may be enlarged after long-term training due to the growth of structures 
supporting nervous function, for example, in the blood vessels that are 
necessary for oxygen and glucose transportation (for a comprehensive review 
see Taubert, Villringer, & Ragert, 2012). 

There are four main reasons why we believe that these effects on brain 
plasticity are more pronounced in music performance than in other skilled 
activities. First, the intensity of goal-directed training is extremely high; 


students admitted to a German state conservatory have spent an average of 


ten years and 10,000 hours of deliberate practice in order to pass the 
demanding entrance examinations (Ericsson et al., 1993). Second, related to 
the above, musical training in those individuals who later become 
professional musicians usually starts very early, sometimes before the age of 
6 when the adaptability of the central nervous system is at its highest. Third, 
musical activities are strongly linked to conditions of high arousal and 
positive emotions, but also to stressors such as music performance anxiety. 
Neuro-active hormones, such as adrenalin (arousal), endorphins (joy), 
dopamine (rewarding experience), and stress hormones (fear of failure) 
support neuroplastic adaptations. Fourth, performing music in public is 
frequently accompanied by strong social feelings best described as a sense of 
connectedness and meaning. As a consequence, increased release of oxytocin 
and serotonin will similarly enhance plastic adaptations (Zatorre & 
Salimpoor, 2013). 

However, we should be careful in claiming that music produces more 
prominent plastic adaptations in the brain compared to other skilled activities 
as the methodology of group comparisons in brain plasticity research might 
produce a bias. For example, group investigations into professional classical 
pianists compared to “non-musicians,” such as in the study by Vaquero et al. 
(2016), might be influenced by differences in sample homogeneity. As 
opposed to many skilled activities, such as playing golf or other sports, or 
other creative professions, such as writing or painting, classical pianists 
experience from a very young age a similar acculturation and take part in 
highly homogeneous activities due to the canonical nature of their training. 
They study similar etudes of Hanon, Czerny, and Chopin for many years, and 
this may well produce more uniform brain adaptations, which dominate any 


individual changes. In other pursuits such as the visual arts, creative writing, 


architecture, jazz improvisation, and music composition, individualized 


training may produce more diverse effects that are masked in group statistics. 


Brain Regions Involved in Performing Music: 
A Quick Overview 


Playing a musical instrument or singing at a professional level requires highly 
refined auditory, sensory-motor, and emotional-communicative skills that are 
acquired over many years of extensive training, and that have to be stored 
and maintained through further regular practice. Auditory feedback is needed 
to improve and perfect performance, and activity of emotion-related brain 
areas is required to render a performance vivid and touching. Performance- 
based music-making therefore relies primarily on a highly developed 
auditory—motor—emotion integration capacity, which is reflected on the one 
hand in increased neuronal connectivity and on the other hand in functional 
and structural adaptations of brain areas supporting these activities. In the 
following, we give a quick overview of the many brain regions involved in 


making music (for a review see Brown et al., 2015). 


Music perception involves primary and secondary auditory areas (A1, 
A2) and auditory association areas (AA) in the two temporal lobes. The 
primary auditory area, localized in the upper portion of the temporal lobe in 
Heschl’s gyrus, receives its main input from the inner ear via the ascending 
auditory pathway. It is mainly involved in basic auditory processing such as 
pitch and loudness perception, perception of time structures, and spectral 
decomposition. The left primary auditory cortex is specialized in the rapid 
analysis of time structures, such as differences in voice onset times when 
articulating “da” or “ta.” The right, on the other hand, deals primarily with 


the spectral decomposition of sounds. The secondary auditory areas surround 


the primary area in a belt-like formation. More complex auditory features 
such as timbre are processed in the secondary auditory areas (Koelsch 2011). 
Finally, in the auditory association areas, auditory gestalt perception takes 
place. Auditory gestalts can be understood, for example, as pitch-time 
patterns like melodies and words. In right-handers, and in about 95 percent of 
all left-handers, Wernicke’s area in the left posterior portion of the upper 
temporal lobe is specialized in language decoding (Kraus, McGee, & Koch, 
1998). 

In contrast to the early auditory processing of simple acoustic structures, 
listening to music is a far more complex task. Music is experienced not only 
as an acoustic structure over time, but also as patterns, associations, 
emotions, expectations, and so on. Such experiences rely on a complex set of 
perceptive, cognitive, and emotional operations. Integrated over time, and 
frequently linked to biographic memories, they enable us to experience strong 
emotions, processed in structures of the limbic system such as the ventral 
tegmental area of the mesencephalon or the accumbens nucleus in the basal 
forebrain (Salimpoor et al., 2013). Memories and social emotions evoked 
during music listening and playing involve the hippocampus, deep in the 
temporal lobe, and the dorsolateral prefrontal cortex, mainly in the right 
hemisphere. 

Making music relies on voluntary skilled movements which involve four 
cortical regions in both hemispheres: the primary motor area (M1) located in 
the precentral gyrus directly in front of the central sulcus; the supplementary 
motor area (SMA) located anteriorly to M1 of the frontal lobe and the inner 
(medial) side of the cortex; the cingulate motor area (CMA) below the SMA 


and above the corpus callosum on the inner (medial) side of the hemisphere; 


and the premotor area (PMA), which is located adjacent to the lateral aspect 


of the primary motor area (see Figure 29.1), 
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Figure 29.1 Brain regions involved in sensory and motor music 
processing. (The abbreviation “a” stands for “area.”) Left hemisphere is 
shown in the foreground (lower right); right hemisphere in the background 
(upper left). The numbers relate to the respective Brodmann’s areas, a 
labeling of cortical regions according to the fine structure of the nervous 
tissue. 


SMA, PMA, and CMA can be described as secondary motor areas, 
because they are used to process movement patterns rather than simple 
movements. In addition to cortical regions, the motor system includes the 
subcortical structures of the basal ganglia, and the cerebellum. Steady 
kinesthetic feedback is also required to control any guided motor action and 
comes from the primary somatosensory area (S1) behind the central sulcus in 
the parietal lobe. This lobe is involved in many aspects of movement 


processing and is an area where information from multiple sensory regions 


converge. In the posterior parietal area, body coordinates in space are 
monitored and calculated, and visual information is transferred into these 
coordinates. As far as musicians are concemed, this area is prominently 
activated during tasks involving multisensory integration, for example, 
during sight-reading, the playing of complex pieces of music (Haslinger et 
al., 2005), and the transformation of musical pitch information into 
movement coordinates (Brown et al., 2013) and of musical notation into 
corresponding motor actions (Stewart et al., 2003). 

The primary motor area (M1) represents the movements of body parts 
distinctly, in systematic order. The representation of the leg is located on the 
top and the inner side of the hemisphere, the arm in the upper portion, and the 
hand and mouth in the lower portion of M1. This representation of distinct 
body parts in corresponding brain regions is called “somatotopic” or 
“homuncular” order. Just as the motor homunculus is represented upside- 
down, so too is the sensory homunculus on the other side of the central 
sulcus. The proportions of both the motor and the sensory homunculi are 
markedly distorted since they are determined by the density of motor and 
sensory innervations of the respective body parts. For example, control of 
fine movements of the tongue requires many more nerve fibers transmitting 
information to this muscle, compared to control of the muscles of the back. 
Therefore, the hand, lips, and tongue require almost two-thirds of the neurons 
in this area (Roland & Zilles, 1996). However, as further explained below, 
the relative representation of body parts may be modified by usage. 
Moreover, the primary motor area does not simply represent individual 
muscles; multiple muscular representations are arranged in a complex way so 
as to allow the execution of simple types of movements rather than the 


activation of a specific muscle. This process is a consequence of the fact that 


a two-dimensional array of neurons in M1 has to code for three-dimensional 


movements in space (Gentner & Classen, 2006). Put more simply, our brain 


does not represent muscles but rather movements. 

The supplementary motor area (SMA) is mainly involved in the 
sequencing of complex movements and in the triggering of movements based 
on internal cues. It is particularly engaged when the execution of a sequential 
movement depends on internally stored and memorized information. It 
therefore is important for both rhythm and pitch processing because of its 
role in sequencing and the hierarchical organization of movement (Hikosaka 
& Nakamura, 2002). Skilled musicians and non-musicians engage the SMA 
either when performing music or when imagining listening to or performing 
music (de Manzano & Ullén, 2012; Herholz & Zatorre, 2012). This finding 
suggests that the SMA may be crucial for experts’ ability to plan music 
segment-by-segment during performance. 

The premotor area (PMA) is primarily engaged when our motor system 
has to react to external stimuli, such as acoustic or visual prompts. 
Anticipation, planning, and preparation of movement patterns in response to 
visual cues have been attributed to the function of PMA (Stetson & 
Anderson, 2015). It is involved in the learning, execution, and recognition of 
limb movements and seems to be particularly concerned with the integration 
of visual information, which is necessary for movement planning. The PMA 
is also responsible for processing complex rhythms (Chen, Penhune, & 
Zatorre, 2008b). 

The function of the cingulate motor area (CMA) is still under debate. 
Electrical stimulation and brain imaging studies demonstrate its involvement 
in movement selection in situations when movements are critical to obtain 


reward or avoid punishment. This fact points toward close links between the 


cingulate gyrus and the emotion processing limbic system. The CMA may 
therefore play an important role in mediating cortical cognitive and limbic- 
emotional functions, for example, in error processing during a musical 
performance (Herrojo-Ruiz, Jabusch, & Altenmiiller, 2009a). 

The basal ganglia, located deep inside the cerebral hemispheres, are 
interconnected reciprocally via the thalamus to the motor and sensory 
cortices, thus constituting a loop of information flow between cortical and 
subcortical areas. They are indispensable for any kind of voluntary action and 
play a crucial role in organizing sequences of motor actions. The basal 
ganglia are therefore the structures mainly involved in automation of skilled 
movements such as sequential finger movements (Seger, 2006). Their special 
function consists of selecting appropriate motor actions and comparing the 
goal and course of those actions with previous experience. The middle 
putamen in particular seems to be involved in storing fast and automated 
movement programs. It is subject to plastic adaptations in professional 
musicians. Furthermore, in the basal ganglia the flow of information between 
the cortex and the limbic emotional systems, in particular the amygdala and 
the accumbens nucleus, converges. It is therefore assumed that the basal 
ganglia process and control the emotional evaluation of motor behavior in 
terms of expected reward or punishment (for a review see Haber, 2003). 

The cerebellum is an essential contributor to the timing and accuracy of 
fine-tuned movements. It is thought to play a role in correcting errors and in 
learning new skills. The cerebellum has been hypothesized to be part of a 
network including parietal and motor cortex that encodes predictions of the 
internal models of these skills. The term “internal model” refers to a neural 
process that simulates the response of the motor system in order to estimate 


the outcome of a motor command. The cerebellum is connected to almost all 


regions of the brain, including those important for memory and _ higher 
cognitive functions. It has been proposed that this structure serves as a 
universal control system that contributes to learning, and to optimizing a 


range of functions across the brain (Ramnani, 2014). 


The Effects of Musical Training on Brain 
Function 


With advanced techniques, brain function can be precisely assessed. Activity 
changes of brain networks, connectivity measures between brain areas on a 
small and a large scale, and even the amount of nerve cells activated in 
response to musical stimuli can be estimated (for a review see Altenmiiller, 
Miinte, & Gerloff, 2004). 


The neural bases of refined auditory processing in musicians are well 


understood. In 1998, Pantev and colleagues provided a first indication that 
extensive musical training can plastically alter receptive functions (Pantev et 
al., 1998). Equivalent current dipole strength, a measure of mass neuronal 
activation, was computed from evoked magnetic fields generated in auditory 
cortex in response to piano tones and to pure tones of equal fundamental 
frequency and loudness. In musicians, the responses to piano tones (but not to 
pure tones) were ~25 percent larger than in non-musicians. In a study of 
violinists and trumpeters, this effect was most pronounced for tones from 


each musician’s own type of instrument (Hirata, Kuriki, & Pantev, 1999). In 


a similar way, evoked neural responses to subtle alterations in rhythm or 


pitch are much more pronounced in musicians than in non-musicians (Miinte, 


Nager, Beiss, Schroeder, & Altenmiiller, 2003). Even functions such as sound 
localization that operate on basic acoustic properties have shown effects of 
plasticity and expertise amongst different groups of musicians. A conductor, 
more than any other musician, is likely to depend on spatial localization for 


successful performance. For example, he might need to guide his attention to 


a certain player in a large orchestra. In one study, professional conductors 
were found to be better than pianists and non-musicians at separating 
adjacent sound sources in the periphery of the auditory field. This behavioral 
selectivity was paralleled by modulation of evoked brain responses, which 
were selective for the attended source in conductors, but not in pianists or 
non-musicians (Miinte, Kohlmetz, Nager, & Altenmiiller, 2001). These 
functional adaptations are not restricted to the auditory cortex, but can be 
observed in subcortical areas of the ascending auditory pathway: musically 
trained individuals have enhanced brainstem representations of musical 
sound waveforms (Wong, Skoe, Russo, Dees, & Kraus, 2007). 

Refined somatosensory perception constitutes another basis of high- 
level performance. The kinesthetic sense is especially important. It allows for 
control and feedback of muscle and tendon tension as well as joint positions, 
which enables continuous monitoring of finger, hand, and lip position in the 
frames of body and instrument coordinates (e.g. the keyboard, the 
mouthpiece). Intensive musical training has also been associated with an 
expansion of the functional representation of finger or hand maps, as 
demonstrated in magnetoencephalography (MEG) studies. For example, the 
somatosensory representation of the left fifth digit in string players was found 
to be larger than that in non-musicians (Elbert, Pantev, Wienbruch, 
Rockstroh, & Taub, 1995). Musicians who had begun training early in life 
(<13 years) demonstrated larger cortical representation of this digit compared 
to those who started to play their instruments later. This finding is reflected at 
a behavioral level in lower two-point discrimination thresholds at the 
fingertips of musicians who started their training earlier (Ragert et al., 2003). 

In motor brain function, changes corresponding to the acquisition of 


musical expertise can also be observed with electrophysiological methods. 


These are mainly related to reduced motor excitability thresholds (Pascual- 


Leone, 2001; Ridding, Brouwer, & Nordstrom, 2000), changes in motor 


receptive fields of trained motor patterns (Pascual-Leone, Grafman, & 
Hallett, 1994), and changes in sensory-motor integration. For example, 
auditory and premotor cortices become co-activated when novices learn to 
play piano. In a longitudinal study, Bangert and Altenmiiller (2003) showed 
that the formation of such multisensory connections between auditory and 
motor areas needs less than six weeks of regular piano training. This finding 
demonstrates how brain adaptations dynamically accompany musical 
learning processes. A _ further causal link between training and 
auditory—motor integration has been shown by findings of enhanced premotor 
recruitment in generating tonal patterns after specific training on the 
production of those patterns (Lahav, Saltzman, & Schlaug, 2007). 

Activation of motor co-representations can occur in trained pianists not 


only by listening to piano tunes (Bangert et al., 2006), but also by observing 


pianists’ finger movements. When pianists observed video sequences of a 
moving hand at the piano, activation was found in additional brain areas as 


compared to musically naive subjects (Haslinger et al., 2005). In addition to 


the hand area in the primary motor cortex, secondary auditory cortices in the 
temporal lobe, and polymodal association cortices in the dorsolateral 
premotor cortex and the parietal cortex were activated. Furthermore, the hand 
areas of the cerebellum were active. This extended neuronal network 
corresponds to a mirror neuron network: a group of functionally connected 
areas involved in imitation of movements and learning through observation 
(Rizzolatti, Fadiga, Gallese, & Fogassi, 1996). As a consequence for musical 
practice, it follows that careful demonstration at the instrument may enhance 


learning. Teaching methods based on demonstration and imitation are widely 


used at all levels of musical training, and would appear to be particularly 
effective in cases where teachers demonstrate an action or series of actions 
that are carefully and methodically observed by the student. 

Practicing through listening and/or observation can be considered as 
special cases of mental training. Narrowly defined, mental training is 
understood as the vivid imagination of movement sequences without 
physically performing them. As with observation of actions, principally the 
same brain regions are active as if the imagined action is performed; that is, 
the primary motor cortex, the supplementary motor cortex, and the 
cerebellum (Kuhtz-Buschbeck et al., 2003). In a study investigating mental 
training of finger movement sequences of different complexities, brain 
activation increased with the degree of difficulty of the imagined motor task. 
Furthermore, when continuing mental practice over a period of several days, 
the involved brain regions showed plastic adaptations. Although these 
adaptations were less dramatic than if the motor tasks were practiced 
physically, mental training produced a clear improvement in_ task 
performance as assessed by finger-tapping tests. 

Many researchers have used functional magnetic resonance imaging 
(f{MRI) to compare neural activities between musicians and non-musicians. 
Differences in activity have been observed across many brain regions when 
individuals are asked to perform musical tasks involving discrimination (e.g. 
Foster & Zatorre, 2010), working memory (e.g. Gaab, Gaser, & Schlaug, 
2006), or production (Bangert et al., 2006, Kleber, Veit, Birbaumer, 
Gruzelier, & Lotze, 2010). Despite the heterogeneity of the tasks used, an 
area that was commonly differentially activated in many of these studies was 
the posterior superior temporal gyrus, which is important for auditory gestalt 


perception, spectrotemporal processing, and auditory-motor transformations 


(Warren, Wise, & Warren, 2005). A recent study identified the left superior 
temporal gyrus as the region that is most linked with musical training, in 
terms of cumulative practice hours (Ellis et al., 2013). As we will see below, 
morphometric studies have found larger amounts of gray matter in this region 
related to expertise and specific auditory skills, such as the possession of 
perfect pitch (Gaser & Schlaug, 2003). 


The Effects of Musical Training on Brain 
Structure 


Since the age of phrenology, neuroscientists have tried to relate extraordinary 
skills to changes in brain anatomy. For example, at the beginning of the 
twentieth century, Auerbach (1906) reported that the middle and posterior 
thirds of the superior temporal gyrus were larger than normal in several 
postmortem studies of the brains of famous musicians. Modern brain-imaging 
techniques such as high-resolution magnetic resonance imaging (MRI), voxel 
based morphometry (VBM), and tensor based morphometry (TBM) allow 
precise determination of gray and white matter volume in predefined brain 
regions. A relatively new technique that can be used to study differences in 
fiber tract volume and direction is diffusion tensor imaging (DTI). This 
provides information about white matter micro-structures by measuring 
diffusion properties of water molecules that move preferentially along the 
myelin sheets of axons. The degree of diffusivity is quantified as fractional 
anisotropy (FA), a measure allowing the assessment of orientation and 


direction of axons and their degree of myelination (Bandettini, 2009). 


In the auditory domain functional adaptations, such as increased 
sensitivity to sounds, are accompanied by anatomical changes in primary or 
secondary auditory cortices of the superior temporal gyrus and the temporal 
plane (Bermudez et al., 2008; Gaser & Schlaug, 2003; Schneider et al., 2005; 
Zatorre, Chen, & Penhune, 2007). A study by Schneider and colleagues 


(2002) in professional musicians, amateurs, and non-musicians is especially 


meaningful. These authors not only found an enlargement of the primary 


auditory cortex (Heschl’s gyrus) related to increased cumulative life practice 
time in professional musicians, but also demonstrated that this enlargement 
was accompanied by more pronounced neuronal representations for pure 
tones in the same region, as reflected in the dipole size of evoked 
neuromagnetic fields. Behavioral tests in the same three groups of subjects 
revealed that the volume of Heschl’s gyrus was positively related to 
improved auditory working memory and_ gestalt perception, as 
operationalized in a musical aptitude test. These behavioral, functional, and 
anatomical changes have been shown to be causally linked to musical 
training in a longitudinal study with children (Hyde et al., 2009). Fifteen 6- 
year-old children received 15 months of piano training and showed not only 
improved auditory perception, but also enlarged gray matter of the right 
primary auditory cortex as compared to 16 age-matched controls. 

In absolute pitch possessors, a pronounced leftward asymmetry of the 
temporal plane was found (Schlaug, Jancke, Huang, & Steinmetz, 1995b). It 
was also demonstrated that in musicians with absolute pitch, the posterior 
superior temporal gyrus is connected to a region within the middle temporal 
gyrus which has been associated with categorical perception (Loui, Li, 
Hohmann, & Schlaug, 2010). Thus, the connections between the posterior 
part of the superior temporal gyrus and the middle temporal gyrus may play a 
role in determining whether or not someone develops absolute pitch, 
alongside early exposure to music. 

In the sensory-motor domain, extensive musical practice during 
childhood and adolescence might have a strong effect on the maturation and 
the development of brain structures involved. Keyboard players have been a 
preferred group to study structural brain changes due to a high demand on 


bimanual dexterity and the possibility of assessing behavior such as speed 


and regularity of finger movements with MIDI-technology (Amunts et al., 
1997; Bangert et al., 2006). In the first study that examined structural 
differences between musicians and non-musicians, Schlaug and collaborators 
(Schlaug, Jancke, Huang, Staiger, & Steinmetz, 1995a) showed that 
professional musicians (pianists and string players) had a larger middle 
section of the corpus callosum compared to a non-musician control group. 
This finding was ascribed to an increase in myelination in the crossing fibers 
of the hand areas of both hemispheres, related to the high demands on 
bimanual coordination. Different research groups using a range of 
methodological approaches have replicated this finding (Gartner et al., 2013; 
Oztiirk, Tascioglu, Aktekin, Kurtoglu, & Erden, 2002, Steele et al., 2013). A 


causal relationship between piano training and enlargement of the corpus 


callosum was established in the abovementioned longitudinal study by Hyde 
et al. (2009). Other fiber tracts have been investigated in musicians: in a 
diffusion tensor imaging (DTI) study with pianists, Bengtsson and colleagues 
(2005) found that the size of several white matter tracts correlated with the 
estimated amount of musical practice during childhood. These structures 
included the posterior limb of the internal capsule, a part of the corticospinal 
tract descending from the motor cortex to the spinal cord, and fiber tracts 
connecting the temporal and frontal lobes. Although the total number of 
practice hours during childhood was lower than in adolescence and 
adulthood, these adaptations support the idea that the central nervous system 
exhibits greater plastic capacities during early stages of development and 
maturation periods. However, some studies have reported lower fractional 
anisotropy in musicians in the corticospinal tract connecting primary motor 
areas with the spinal cord (Imfeld, Oechslin, Meyer, Loenneker, & Jancke, 


2009), and in the arcuate fasciculus, the fiber tract connecting auditory and 


premotor regions (Halwani, Loui, Rtiber, & Schlaug, 2011). According to 
Schlaug (2015) these discrepant results may be explained by the fact that 
these fiber tracts are aligned in a less parallel manner than in non-musicians 
due to increased axonal sprouting and more branching of axons. In future, 
imaging technologies may provide a more fine-grained picture of nervous 
tissues. 

Concerning the size of primary motor cortex, various findings have been 
reported. In pianists, the depth of the central sulcus, often used as a marker of 
primary motor cortex size, was larger in both hemispheres but compared to 
non-musicians more pronounced on the right hemisphere, corresponding to 
the non-dominant left hand function (Amunts et al., 1997; Schlaug, 2001). It 
was argued that years of manual motor practice of the non-dominant left hand 
produced this effect on the right hemisphere. For the dominant right hand and 
left hemisphere this effect was believed to be masked, since it undergoes 
some form of fine-motor training in everyone who writes and performs other 
skilled sensory-motor tasks with that hand. As was observed for the corpus 
callosum, there was a positive correlation between the size of the primary 
motor cortex and the onset of instrumental musical training. Again, a causal 
relationship was established in the abovementioned longitudinal study in 
child piano novices, with an increase in gray matter density in the right motor 
hand area associated with 15 months of piano training (Hyde et al., 2009). A 
recent investigation into middle-aged pianists revealed some interesting 
details concerning the effect of ongoing expertise on lifelong plasticity 
(Gartner et al., 2013). Pianists who continued to give concerts and practice 
for a minimum of three hours a day showed not only larger motor hand areas, 
but also larger foot areas in the sensory-motor cortices of both hemispheres 


than pedagogues who had majored in piano performance, but who had 


practiced for less than two hours a day over the last ten years. This result 
relates to the important role of pedaling in piano performance. Pedaling is a 
highly refined skill requiring spatiotemporal control in the range of 
millimeters and milliseconds in order to adaptively modulate color, 
expressivity, and loudness of the music. 

Structural brain differences have been reported in musicians who play 
different instruments (Bangert & Schlaug, 2006). For keyboard players, the 
omega-shaped folding of the precentral gyrus, which is associated with hand 
and finger movement representation, was found to be more prominent in the 
left hemisphere for keyboard players, but was more prominent in the right 
hemisphere for string players. This structural difference is likely to reflect an 
adaptation to the specific demands of different musical instruments. 
Obviously, the rapid and spatiotemporally precise movements of the left hand 
in string players are a stronger stimulus for plastic adaptations compared to 
the right hand bowing movements requiring the fine-tuned balance of fingers 
at the frog of the bow, and precise movements of wrist and arm. Gaser and 
Schlaug (2003) compared professional pianists, amateur musicians, and non- 
musicians and reported increased gray matter (GM) volume in professional 
musicians not only in primary motor, somatosensory, and premotor areas, but 
also in multisensory parietal integration areas and in cerebellar brain regions. 
Modeling musical expertise with the same three-group population, James and 
collaborators (James et al., 2013) reported an intricate pattern of 
increased/decreased GM. In particular, musicians showed GM _ density 
increases in areas related to higher-order cognitive processes (such as the 
fusiform gyrus or the inferior frontal gyrus), whereas GM decreases were 
found in sensory-motor regions (such as perirolandic and striatal areas). 


These reductions in GM were interpreted as reflecting a higher degree of 


automaticity of motor skills in more expert musicians. Like gray matter, 
white matter differs between different instrumentalists, for example, a larger 
FA at the corpus callosum for the string players than the pianists (Vollmann 
et al., 2014). 

It is now well established that along with increasing expertise, not only 
enlargement but also reduction of neural structure can be observed. This was 
first established in a study of pianists targeting the middle putamen in the 
basal ganglia: a brain region involved in automation of motor programs. 
Granert and colleagues (Granert, Peller, Jabusch, Altenmiiller, & Siebner, 
2011) measured the skill level of piano playing via temporal accuracy in a 
scale-playing task. These authors found that the higher the level of piano 
playing, the smaller the volume of gray matter in the right putamen. This 
reduction was ascribed to an optimization process of neuronal networks 
within the putamen, leading to fewer, but more efficient and stable dendritic 
and axonal connections in this area of the motor basal ganglia loop. 

Until recently, it remained an open question as to the degree to which 
these structural and functional brain changes are influenced by age at onset of 
musical activity and by cumulative practice hours over particular periods of 
life. These factors have often been confounded and it was generally believed 
that early commencement of musical activity, along with increased life 
practice time, resulted in enlarged neural representations underpinning 
auditory or sensory-motor skills. Steele and colleagues (2013) were the first 
to investigate the morphology of the corpus callosum such that they could 
compare its white matter organization in early- and late-trained musicians 
who had been matched for years of training and experience. They found that 
early-trained musicians had greater connectivity in the posterior part of the 


corpus callosum and that fractional anisotropy in this region was related both 


to age at onset of training and to sensory-motor synchronization performance. 
They concluded that training before the age of 7 resulted in changes in white 
matter connectivity that may serve as a scaffold upon which ongoing 
experience can build. Inspired by this work, and since in this study neither 
gray matter density nor the size of specific brain areas was analyzed, we 
designed a similar brain morphometry study in a group of 36 award-winning 


professional pianists (Vaquero et al., 2016). We kept cumulative life practice 


time constant, but split the group into 21 pianists who had started their 
musical training before age 7, and another group of 15 who had started after 
that age. We compared brain anatomy between these groups, and between 
musicians and age-matched medical students who were non-musicians. In 
addition, 28 pianists from the sample completed a scale-playing task, in order 
for us to obtain an objective measure of their pianistic abilities and temporal 
precision. Compared with non-musicians, pianists showed more gray matter 
in regions associated with learning (hippocampus), sensory and motor control 
and processing (putamen and thalamus), emotional processing and the reward 
system (amygdala), and with auditory and language processing (left superior 
temporal cortex). However, they also showed less gray matter in regions 
involved in sensory and motor control (postcentral gyrus) and processing of 
musical stimuli (right superior temporal cortex), as well as structures that 
have been related to music score reading (supramarginal gyrus). Moreover, 
among the pianists it was observed that the size of the right putamen 
correlated significantly with the age at which music training began: the 
earlier they started to play the piano, the smaller the volume of gray matter in 
the right putamen (see Figure 29.2). In keeping with the interpretation of the 
results of Granert et al. (2011) reported above, pianists who started earlier in 


life optimized functionality of neural structures involved in sensory-motor 


processing, motor learning, and motor memory. This is reflected in the 
behavioral task: those pianists who had started their musical training before 
age 7 played with higher regularity than those who started after that age, even 
though all of the pianists practiced for the same number of hours around the 
time of the study and had achieved the same level of proficiency. This is an 
important scientific proof of common knowledge, expressed in the German 
proverb “a tree must bend while it is young.” Neuroscientifically it is an 
interesting phenomenon, showing that even for highly complex motor tasks, 
sensitive periods in the nervous system exist (Furuya et al., 2014a). However, 
as we have seen, such windows of opportunity can depend on domain, 


genetics, and continuing training. 
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Figure 29.2 Summary of the results of the study on pianists by Vaquero et 
al. (2016). 


Courtesy of Lucia Vaquero and Antoni Rodriguez-Fornells (with 


permission). Explanations are given in the text. 


De-expertise: Musician’s Dystonia as a 
Syndrome of Maladaptive Plasticity 


Brain plasticity is not always beneficial. Overtraining, fear of failure, chronic 
pain, and other stressors may trigger a deterioration of motor control and 
initiate a process of degradation of skill. We propose to term this plasticity- 
induced loss of skills “de-expertise.” Approximately one or two in 100 
professional musicians suffer from a loss of voluntary control of their 
extensively trained, refined, and complex sensory-motor skills. This 
condition is generally referred to as focal dystonia, violinists’ cramp, or 
pianists’ cramp. In many cases, focal dystonia is so disabling that it 


prematurely ends the artist’s professional career (Altenmiiller, Ioannou, & 


Lee, 2015). The various symptoms that can mark the beginning of the 
disorder include subtle loss of control in fast passages, finger-curling (cf. 
Figure 29.3), lack of precision in forked fingerings in woodwind players, 
irregularity of trills, fingers sticking on the keys, involuntary flexion of the 
bowing thumb in strings, and impairment of control of the embouchure in 
woodwind and brass players in certain registers. At this stage, most musicians 
believe that the reduced precision of their movements is due to a technical 
problem. As a consequence, they intensify their efforts, but this often only 


exacerbates the problem. 


Figure 29.3 Typical patterns of dystonic posture in a pianist, violinist, 


trombonist, and flautist. 


Musician’s dystonia (MD) has been described for almost every 
instrument, including keyboard, strings, plucked instruments, woodwind, 
brass, percussion, and folk instruments such as bagpipes and accordion. In a 
recent study of the epidemiology of 369 German professional musicians 
suffering from MD, keyboard players were most common with 27.1% 
(percentage in the healthy musician population 29%; Altenmiiller, Baur, 
Hofmann, Lim, & Jabusch, 2012), followed by woodwinds (21.7%, vs. 15% 


healthy musicians) and brass players (20.9%, vs. 10% healthy musicians). 


When subdividing these instrumental groups into single instruments, piano 
represented 22% of the total, guitar 15.2%, flute 9.7%, and violin 7.6% (Lee, 
Heifs, Eich, Ioannou, & Altenmiiller, forthcoming). 

The relative rarity of certain instruments in these studies (even after 
accounting for the relative number in the healthy musician population), such 
as double bass, harp, and cello, and the overrepresentation of piano, guitar, 


and flute suggests that the specific physical demands of the instrument 


constitute risk factors (Altenmiiller et al., 2012). For example, fast, fine, and 
simultaneous finger placement in guitar, piano, violin, and flute may be more 
likely to trigger the development of dystonia compared to forceful finger 
flexion at a lower tempo as required in double bass and cello. It is likely that 
the distribution of affected hands in musician’s dystonia is not random. 
Among pianists and guitarists the right hand is more frequently involved, 
among violinists the left hand. These trends correlate with differences in the 
technical demands imposed by each instrument. The virtuoso piano repertoire 
demands more from the right hand than the left, as the right hand must 
usually play faster trills, arpeggios, and ornaments. It also often carries the 
treble melodic line with the third, fourth, and fifth fingers (Lehmann & 
Ericsson, 1998). On the violin the technical demands of the left fingering 
hand are more pronounced, requiring temporo-spatial precision in the 
millisecond and millimeter range for professional-level playing. Among 
woodwind players, both hands are equally likely to be affected. An exception 
is in flautists, for whom the left hand is more commonly problematic, due to 
the fact that the left index finger must support the weight of the instrument as 
well as depress the keys, and that the left thumb, ring and little fingers are 
extremely active compared to those of the right hand (Baur, Jabusch, & 
Altenmiiller, 2011). 

The interplay of predisposing and triggering factors in musician’s 
dystonia has become a topic of intense research (Jabusch & Altenmiiller, 
2004; Jabusch, Miiller, & Altenmiiller, 2004; Ioannou & Altenmiiller, 2014; 
Ioannou, Furuya, & Altenmiiller, 2016). As predisposing factors, male gender 
and genetic susceptibility to malfunction of neuronal networks involved in 
sensory-motor representations and pathways play a role (Schmidt et al., 


2009). Furthermore, psychological traits such as elevated anxiety and 


obsessive-compulsive-disorder (OCD) have been documented. These may be 
partly interrelated: male gender and genetic susceptibility for malfunction of 
sensory-motor pathways may have a common origin, such as certain genes 
located on the X-chromosome. OCD and anxiety can in turn lead to dystonia- 
triggering behaviors, such as overuse of muscles and exaggerated, repetitive 
training. The following triggering factors have been identified: high motor 
demands; extra-instrumental activities, such as writing and typing (Baur et 
al., 2011); late onset of training; playing classical music with high demands 
on precise temporo-spatial control; increased general psychological and 
muscular tension; and destabilization of overlearned sensory motor programs 
due to imposed technique changes or sensory disturbances following nerve or 
soft-tissue injury (for a review, see Altenmiiller et al., 2015). The interplay of 
predisposing and triggering factors in the development of musician’s dystonia 
is depicted in Figure 29.4. According to our model, the emergence of 
musician’s dystonia is a gradual process, starting with reversible tensions and 
dysfunctional movements termed “dynamic stereotypes,” which are then 
gradually consolidated in sensory-motor memories and finally automated. 
Only at this stage would we speak of musician’s dystonia, which is defined as 


a persistent muscular incoordination that is difficult to cure. 
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Figure 29.4 Heuristic model demonstrating the proposed interplay of 
sensory-motor and psychological triggering factors and their contribution 
to the different motor disturbances. The assumed degree of 
“psychological” triggering factors is displayed as the dotted space; the 
increasing gray shade symbolizes the increasing degree of loss of motor 


control. 


With respect to predisposing factors, demographic data demonstrate a 
preponderance of male musicians with a male/female ratio of about 4:1. 
When compared to the gender distribution in healthy musicians, analyzed 
according to the instrumental groups, male musicians were more likely to 
have musician’s cramp than females in keyboard, string, woodwind, and 
plucking instruments. The only instrumental group without a gender bias for 
symptoms was the brass instruments (Lim & Altenmiiller, 2003). 

Hereditary factors play an important role, since a positive family history 


of dystonia in first-degree relatives can be found in up to 36 percent of 


affected musicians (Schmidt et al., 2009). Premorbid OCD and anxiety 
disorders are found in 40 percent of our patients (Ioannou & Altenmiiller, 
2014; Ioannou et al., 2016). The mean age at onset of dystonic symptoms is 
35 years (Lee et al., forthcoming); however, those musicians with elevated 
anxiety and stress levels develop dystonia on average ten years earlier 
(Ioannou et al., 2016). Furthermore, there is high variability, with the 
youngest patients developing dystonia at the age of around 18 and the oldest 
while in their sixties (Altenmiiller, 2003). 

Late onset of training is another important triggering factor. Musicians 
with dystonia start their training at an average age of around 10, whereas the 
mean starting age for healthy musicians is 7 (Schmidt, Jabusch, Altenmiiller, 
Kasten, & Klein, 2013). This underlines the important role of early 
optimization of sensory-motor programs to allow the establishment of a 
stable scaffold for further improvements over the lifetime. The term 
metaplasticity, referring to sensitive periods in the lifetime during which 
general learning strategies in specific domains are best acquired, describes 
this phenomenon (for a review, see Altenmiiller & Furuya, 2016a). 
Metaplasticity may also account for the fact that alteration of playing 
technique at a later age, for example when changing teacher during the 
transition from school to university, may destabilize motor programs and 
trigger dystonia. 

The genre in which the overwhelming majority of patients suffering 
from focal dystonia operate is classical music. In contrast to pop or jazz, with 
improvised structures and great freedom of interpretation, musical constraints 
are most severe in classical music. This genre requires maximal temporal 
accuracy (in the range of milliseconds), which is scrutinized by the 


performing musician as well as by the audience throughout the performance. 


As a consequence, public performances of classical music can involve a high 
level of social pressure; here the gap between success and failure is minimal. 

Finally, physical and psychological traumata are frequently found in the 
history of musicians suffering from focal dystonia. With respect to physical 
traumata, we know that sensory disturbances and chronic pain may induce 
maladaptive plastic changes in the brain’s sensory and motor networks, 
leading to a degradation of motor control. This is best documented in neural 
studies of the consequences of complex regional pain syndromes (Henry, 
Chiodo, & Yang, 2011). Research on psychological trauma triggering 
musician’s dystonia is in its infancy. Jabusch and Altenmiiller (2004) 
proposed that acute and chronic stress may promote dysfunctional motor 
memory formation via beta-adrenergic activation, stress hormone release, 
increased muscle tone, and associative learning. 

In summary, the association between the abovementioned trigger factors 
and the development of focal task-specific dystonia is highly suggestive of a 
strong environmental contribution to the pathogenesis of musician’s dystonia. 
However, the observation that the majority of performing artists fortunately 
do not develop musician’s dystonia underlines the important role of 


underlying genetic predispositions. 


Brain Changes Associated with Loss of 
Sensory Motor Control 


The etiology of focal dystonia is not completely understood at present, but is 
probably multifactorial. Most studies of focal dystonia reveal abnormalities 
in three main areas: (a) reduced inhibition in the motor system at cortical, 
subcortical, and spinal levels; (b) altered sensory perception and integration; 
and (c) impaired sensory-motor integration. All of these changes are believed 
to primarily originate from dysfunctional brain plasticity. 

A lack of inhibition is a common finding in studies of patients with all 


forms of dystonia (for a review, see Lin & Hallett, 2009). Fine motor control 


in general requires a subtle balance in neural circuits between excitation and 
inhibition. This fact is particularly important in allowing precise and smooth 
hand movements. For example, rapid individuated finger movements in piano 
playing require selective and specific activation of muscles to move the 
intended finger in the desired manner, and to inhibit movements of 


uninvolved fingers (Furuya, Oku, Miyazaki, & Kinoshita, 2015). In patients 


suffering from hand dystonia, electromyographic recordings have revealed 
abnormally prolonged muscle firing with co-contraction of antagonistic 
muscles and overflow of activation to inappropriate muscles (Furuya & 


Altenmiiller, 2013a, 2013b). Lack of inhibition is found at multiple levels of 


the nervous system. At the spinal level, it leads to reduced reciprocal 
inhibition of antagonistic muscle groups producing co-contraction, for 
example of wrist flexors and extensor muscles. This in turn produces a 


feeling of stiffness and immobility, and frequently leads to abnormal postures 


with predominant flexion of the wrist due to the relative strength of the flexor 
muscles. Abnormal inhibition has also been demonstrated at the cortical level 
by using non-invasive transcranial magnetic stimulation to measure 
intracortical inhibition (Sommer et al., 2002). Interestingly, at this level 
abnormal inhibition is frequently seen in both hemispheres, despite unilateral 
symptoms. This points toward a more generalized form of inhibition deficit. 
Finally, lack of inhibition is also seen in more complex tasks, such as when 
movement preparation is required prior to scale playing, and for sudden 
movement inhibition following a stop signal in pianists (Herrojo-Ruiz et al., 
2009b). The ubiquitous demonstration of deficient inhibition is suggestive of 
a common underlying genetic cause. However, it has to be emphasized that 
none of these electrophysiological effects allows diagnosis on an individual 
level, since the variability in both healthy and dystonic musicians is 
extremely large. 

Altered sensory perception may also be a sign of maladaptive brain 
plasticity. Several researchers have demonstrated that the ability to perceive 
two stimuli as temporally or spatially separate is impaired in patients with 
musician’s dystonia, whether sensation is via the fingertips (in hand 
dystonia), or the lips (in embouchure dystonia). This behavioral deficit is 
mirrored in findings of the cortical somatosensory representation of fingers or 
lips. It has been demonstrated with various functional brain imaging methods 
that in somatosensory cortex the topographical locations of sensory inputs 
from individual fingers overlap more in patients with musician’s cramp than 


in healthy controls (Elbert et al., 1998). Similarly, lip representation may be 


altered in patients suffering from embouchure dystonia (Haslinger, 
Altenmiiller, Castrop, Zimmer, & Dresel, 2010). Other abnormalities include 


elevated temporal discrimination thresholds, a marker of basal ganglia 


dysfunction found to be relevant to the pathogenesis of focal dystonia 
(Termsarasab et al., 2015). Since in healthy musicians an increase of the size 
of sensory finger representations has been interpreted as an adaptive plastic 


change to support the current needs and experiences of the individual (see 


above, Elbert et al., 1995), it could be speculated that these changes over- 
develop in musicians suffering from dystonia, shifting brain plasticity from 
being beneficial to maladaptive (Rosenkranz et al., 2005). In this context it is 
worth recalling that local pain and intensified sensory input due to nerve 
entrapment, trauma, or muscle overuse have been described as potential 
triggers of dystonia. There are clear parallels of abnormal cortical processing 
of sensory information and cortical reorganization between patients with 
chronic pain and those with focal dystonia. An animal model of focal 
dystonia established in over-trained monkeys supports this suggestion; 
repetitive movements induced both types of symptoms — pain syndromes as 
well as dystonic movements. Mapping of neural receptive fields has 
demonstrated a distortion of cortical somatosensory representations (Byl, 
Merzenich, & Jenkins, 1996), suggesting that over-training and practice- 
induced alterations in cortical processing may play a role in focal hand 
dystonia. 

Impaired sensory-motor integration also plays an important role in the 
pathophysiology of musician’s dystonia. This is best illustrated by the 
“sensory trick” phenomenon: some musicians suffering from dystonia show a 
marked improvement of fine motor control when playing with a latex glove, 
or when holding an object (such as a rubber gum) between the fingers, thus 
changing the somatosensory input. In experimental settings, vibrating stimuli 
led to a worsening of musician’s dystonia. In one study, when transcranial 


magnetic stimulation was used in conjunction with muscle vibration, motor 


evoked potentials decreased in agonist muscles and increased in antagonist 


muscles (Rosenkranz, Altenmiiller, Siggelkow, & Dengler, 2000). These data 


again suggest an altered central integration of sensory input in musician’s 
dystonia, which might be due to the failure to link the proprioceptive input to 
the appropriate motor cortical area. Reversing these effects of sensory-motor 
disintegration is the approach of several retraining therapies. Sensory 
retraining in the form of tactile discrimination practice can ameliorate motor 
symptoms, suggesting that the abovementioned sensory abnormalities may 
drive the motor disorder. Interestingly, a positive response to the sensory 
trick phenomenon is linked to a better outcome in attempts to re-educate 
musicians with dystonia (Paulig, Jabusch, Gro&bach, Boullet, & Altenmiiller, 
2014). 

Innovative brain imaging techniques recently demonstrated that 
musician’s dystonia is also a network disorder. Assessment of changes in 
functional brain networks and in neural connectivity between different brain 
regions revealed that patients with musician’s dystonia show altered network 
architecture characterized by abnormal expansion or shrinkage of neural cell 
assemblies. These changes include the breakdown of basal ganglia—cerebellar 
interaction, loss of a pivotal region of information transfer in the premotor 
cortex, and pronounced reduction of connectivity within the sensory-motor 
and frontoparietal regions (e.g. Battistella, Termsarasab, Ramdhani, 
Fuertinger, & Simonyan, 2015; Striibing, Ruiz, Jabusch, & Altenmiiller, 
2012). These abnormalities have been further characterized by significant 
connectivity changes in the primary sensory-motor and inferior parietal 
cortices. Therefore, musician’s dystonia likely represents a disorder of large- 
scale functional networks. However, the specific role of these networks and 


their inter-individual variability remain to be clarified. 


There are currently several treatment methods available for musician’s 
dystonia. Novel strategies aim to reverse the maladaptive plastic changes, for 
example with inhibition of overactive motor areas on the affected side, 
alongside activation of the contralateral “healthy” motor cortex, whilst 
musicians perform in-phase symmetrical finger exercises on a keyboard 
(Furuya, Nitsche, Paulus, & Altenmiiller, 2014b). Retraining may also be 
successful, but usually requires several years to succeed (van Vugt, Boullet, 


Jabusch, & Altenmiiller, 2014). Symptomatic treatment through temporary 


weakening of the cramping muscles by injecting Botulinum toxin has proven 
to be helpful in other cases; however, since the injections need to be applied 
regularly every three to five months during one’s career, this approach does 
not offer a good solution for young patients. Thus, the challenge is to prevent 
young musicians from acquiring such a disorder. The components of such a 
prevention program include reasonable practice schedules, economic 
technique, prevention of muscle overuse and pain, mental practice, avoidance 
of exaggerated perfectionism, and psychological support with respect to self- 


confidence. 


Brain Plasticity as Prerequisite and Result of 
Expert Performance in Musicians 


In the preceding paragraphs, we have demonstrated how musical activities, 
such as learning to master an instrument and to perform in public, induce 
brain plasticity. These central nervous adaptations are in most cases 
beneficial but in some circumstances may be detrimental, as illustrated in 
musician’s dystonia. Age at commencement of practice, amount and quality 
of practice, genetic predisposition, and accompanying conditions, such as 
stressors or muscular overuse, determine the quality and nature (adaptive or 
maladaptive) of these brain changes. Furthermore, the brain’s “sensitive 
periods,” when it is best shaped, seem not only to depend on hereditary 
factors but also to vary in different sensory, motor, and cognitive domains. 
We propose the concept of metaplasticity, conceptualized above with a 
scaffold metaphor: early musical training stabilizes the sensory-motor system 
and provides neuroprotective effects with respect to the development of focal 
dystonia (see Figure 29.5). These effects are maintained for the whole 
lifespan: those musicians who start early not only develop superior auditory 
and sensory-motor skills, they also show less age-induced decline of sensory- 


motor and cognitive functions (Krampe & Ericsson, 1996; Meinz, 2000). 


Thus, intense musical training in childhood can bring about lifelong change 
in both structure and functions of auditory, sensory-motor, and emotional 
systems. These not only enhance musical skill acquisition and guard against 
disorders triggered by extensive training, but also serve as ingredients for 


better shaping lifelong neuronal development. 


Metaplasticity in musicians 


Late start of musical skill acquisition (after age 7) 
Musicians suffering from dystonia 


Genetics : Boosting of the Maintenance & Age-dependent 
Sensory-motor skills effects of improvement cessation of daily 
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Figure 29.5 Different time courses of skill acquisition in earlier and later 
“starters,” and in musicians suffering from dystonia. Neuronal networks 
underpinning specific skills are optimized in childhood. This allows more 
effective acquisition during pre-adolescence and adolescence. Skill level 
continues to improve during adulthood, and stabilizes as related activities 
cease. Early-optimized neuronal networks are more stable and less 
susceptible to maladaptive changes, such as occur in musician’s dystonia. 
Here late inception of training and specific trigger factors may lead to a 


deterioration of sensory-motor skills. 


We would like to conclude our chapter with a general remark. As 
emphasized above, the complex neurophysiological processes involved in 
musical training and expert performance are not restricted to sensory-motor 
brain circuits, but also involve memory, imagination, creativity, and — most 
importantly — emotional communicative skills. The most brilliant virtuosos 
will not move their listeners if imagination, color, fantasy, and emotion are 


not part of their artistic expression. These qualities are often not trained 


solely within a practice studio, but depend on and may be linked to 
experience from daily life, human relationships, a rich artistic environment, 
and emotional depth. Such factors that profoundly influence the aesthetic 
quality of music performance can be subject to expertise research; however 
they are presently inaccessible to neuroscientific methodology. Important 
steps here will include the development of more fine-grained imaging 
technologies and the integration of findings from brain morphology, nerve- 
cell metabolism, connectivity measures, and neurotransmitter activities, at the 
individual level (Amunts & Zilles, 2015). Along with experimental 
paradigms that include meaningful behavioral measures, such research may 
eventually enable us to uncover the secrets of musical creativity and its 


emotional power. 
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Introduction 


Visual art is one of humanity’s great achievements (Murray, 2003). The 
archaeological evidence of prehistoric art, most gloriously manifested in the 
cave paintings of Chauvet, Lascaux, and Altamira, suggests Homo sapiens 
has engaged in visual art for the entirety of our existence as a species, 
certainly since the so-called “creative explosion” (Pfeiffer, 1982) some 
30,000 years ago. In historical times, great artists like Michelangelo have 
been accorded quasi-divine status (Vasari, 1996), and contemporary museum 
attendance and the prices of notable paintings at auction bespeak art’s 
continued significance. Decorative visual art is a ubiquitous outlet of creative 
expression, appearing in every known human culture (Brown, 1991). Visual 
artistry has likewise been identified as a fundamental domain of the mind 
(Feist, 2004; Gardner, 1983). 

Art’s importance stems partly from the fact that humans are 
predominantly visual creatures. Much of the brain is involved in efficiently 
processing diverse streams of visual information (including form, color, and 
motion) to establish a stable, interpretable percept despite ambiguous, 
transient, or incomplete input (Palmer, 1999). Artworks involve another kind 
of visual degradation; even highly realistic images, in which artists attempt to 
mimic what they see as closely as possible, entail significant information loss 
compared to the real world — with a concomitant set of choices on the part of 
the artist about what to depict and how to depict it (Gombrich, 1960). 
Throughout history, visual artists have exploited many technical devices to 


facilitate the perception and recognition of the content of images, including 


means of rendering contours, depth cues, and illumination (Melcher & 
Cavanagh, 2011). 

That the modality of vision is so central to art raises a potentially thomy 
problem in demarcating and characterizing visual art as a self-contained 
domain of expertise. Performance differences in art are obvious. 
Accomplished artists can create strikingly convincing depictions of visual 
scenes, while most adults find even rudimentary aspects of observational 
drawing difficult. Historically, artists as well as scholars across many 
domains have attributed variability in drawing skill to differences in visual 
perception, encapsulated in the claim that artists see the world differently 
from non-artists (Kozbelt & Seeley, 2007). Over the last few decades, 
psychologists have conducted substantial empirical research on this issue, 
advancing several categories of psychological explanations. 

In this chapter, we review current psychological research on artistic 
expertise, exploring the evidence suggesting possible perceptual differences 
between artists and non-artists via two venerable and prominent theoretical 
accounts, which we term the “bottom-up” and “top-down” explanations. Our 
discussion emphasizes the skills required for the production of realistic 
depictions, rather than merely receptive aspects like appreciation or 
connoisseurship. While researchers have produced some interesting findings 
on artistic expertise, many conceptual and methodological issues remain 
unresolved, and we conclude the chapter by discussing some of these. The 
crucial issue relevant to the classic expertise literature is the extent to which 
depictive skill may entail robust, general advantages in perception and 
attention, which, beyond simply reflecting an acquired body of domain- 
specific patterns, represent artists’ enhanced ability to solve the same kinds of 


problems as the human visual system does generally. 


Making Representational Depictions 


Understanding the nature of expertise in visual art requires understanding the 
nature of artists’ activity. While there are many modes of artistic expression, 
we focus on the observational rendering of realistic two-dimensional images 
(especially drawing) as paradigmatic of many of the most interesting aspects 
of visual artistry. Realistic observational drawing involves creating a 
depiction of an external model stimulus with the goal of achieving visual 
accuracy. A visually accurate rendering is “one that can be recognized as a 
particular object at a particular time and in a particular space, rendered with 
little addition of visual detail that cannot be seen in the object represented or 
with little deletion of visual detail” (Cohen & Bennett, 1997, p. 609). Given 
the visual system’s organization, whereby numerous streams of visual input 
are processed in parallel, there are many ways of achieving visual realism 
across media and styles, all of which are to some degree artificial — that is, 
based on invented techniques that capture only a subset of visual experience 


(Gombrich, 1960; Willats, 1997). Contrast the paintings of masters like Jan 


van Eyck and Diego Velazquez, which are virtually non-overlapping in terms 
of specific aspects of style, yet share a profound (if intuitive) fidelity to the 
visible world. 

Irrespective of style, observational drawing entails intense and 
prolonged perceptual engagement, with plausible long-term ramifications on 
artists’ basic perceptual and attentional capabilities. Consider, in information 
processing terms, the seemingly simple act of drawing a pear from 


observation. One must translate a fleeting iconic sensation of the pear’s 


appearance into a longer-term representation to understand its relative 
proportions and three-dimensional structure, as observed from a particular 
viewpoint. One must attend to and make decisions about the key points of 
concavity on the surface of the pear to establish a proportional framework, 
engage a motor program to execute those marks on the paper, and then assess 
the accuracy of those marks with an eye to potentially revising them. Once 
proportions are roughly established, how should the pear’s form be 
conveyed? Outline? (How heavy? Continuous or broken? If broken, where?) 
Shading with the side of the pencil lead? (Where to begin and end? How to 
modulate the tone?) Shading using lines? (Parallel, cross-hatched, or 
haphazard? Straight or curved? At what angle relative to the light source? 
Deployed to emphasize discrete planes? Should darkness be established by 
denser line allocation or pressing harder on the pencil? How should one 
distinguish the pear’s mottled texture and local color versus shading due to 
the play of light?) Clearly, even drawing a simple object entails navigating 
many decisions, implicitly or explicitly, with the constituent perceptual and 
attentional processes representing a specialized mode of engagement — one 
which, while utilizing many of the same fundamental mechanisms, seems far 
removed from everyday processes used for navigation and _ object 
categorization (Kozbelt & Seeley, 2007). 

The supposed distinction between everyday perception and whatever 
special mode of perception might best capture artists’ expertise is a debatable 
and multifaceted issue. Is artistic expertise rooted in very specific classes of 
stimuli with which artists have direct experience (like faces in portraiture), in 
specialized knowledge of artistic styles they might deploy (like realism or 
Impressionism), in the use of particular media of artistic expression (like 


charcoal or oil paint), or in the actual experience of producing art? To what 


extent do specifically aesthetic or creative modes of cognition (e.g. Cupchik, 


1992; Martindale, 1990), which may differ substantially from both everyday 


perception and a mode of perception emphasizing visual realism, contribute 
to artists’ expertise? In terms of visual processing advantages, is the expertise 
of artists better characterized as domain-specific (i.e. tied to particular 
categories of familiar stimuli) or domain-general (i.e. applying to visual 
processing in more general, flexible ways)? 

Despite a recent surge in research on artists and perception, many of 
these questions defy simple answers. Constrained by the nature of extant 
research, here we stake out the following positions vis-a-vis these questions. 
First, we focus on the necessity of having experience in making art for 
perceptual advantages to accrue, particularly since most researchers have 
tested visual artists, rather than art critics or historians (thus providing little 
guidance on possible perceptual advantages among members of these latter 
groups). Moreover, while knowledge of the effects of particular artistic media 
likely constitutes an important aspect of real-world artistic expertise (Kozbelt 


& Seeley, 2007), most laboratory drawing tasks involve just pencil and paper, 


minimizing the relevance of media-specific knowledge in empirical studies. 
Second, we focus exclusively on drawing tasks involving accurate, visually 
realistic depictions, where creativity is often explicitly discouraged; thus, the 
extent to which visual accuracy itself may be indirectly guided by specifically 
aesthetic or creative modes of processing is unclear. Third, in terms of 
domain-specificity versus domain-generality, we argue that artists’ perceptual 
advantages show greater flexibility than in many standard pattern-matching 
accounts of expertise (as in chess — Chase & Simon, 1973), though this need 
not extend to very early stages or very low levels of visual processing. This 


latter point is perhaps most relevant to characterizing the nature of artists’ 


perceptual expertise along theoretical lines in the classic expertise literature 
(e.g. Gobet & Chamess, Chapter 31, this volume; Landy, Chapter 10, this 
volume). 

Whatever the characterization of artists’ mode of perception, artists 
themselves have often stressed the importance of perceptual factors in 
depictive skill. Leonardo remarked that “a painter ought always to have in 
mind a kind of routine system to enable him to understand any object that 
interests him” (Kelen, 1990, p. 23). Cézanne described his own system for 
simplifying objects’ forms by noting, “nature must be treated through the 


cylinder, the sphere, the cone” (Goldwater & Treves, 1972, p. 363). Ingres 


advised artists, “draw with your eyes when you cannot draw with a pencil” 
(Goldwater & Treves, 1972, p. 217). Consistent with these remarks, a 
questionnaire study of contemporary artists (Schlewitt-Haynes, Earthman, & 
Burns, 2002) suggests that from an early age they are highly involved with 
visually analyzing the world. 

As Landy (Chapter 10, this volume) emphasizes, gaining expertise in 
complex domains crucially requires learning to see well. Across different 
domains, perceptual processes can play out in remarkably varied ways, 
including attending to important aspects of problems to control the time 
course of reasoning and performing important conceptual tasks by offloading 
them onto perceptual-motor processing networks. Among the aspects of 
perceptual learning most relevant to visual art are tuning perception and 
attention, in order to highlight visual features relevant to a particular task, and 
learning appropriate perceptual skills. How artists might engage perceptual 
and attentional mechanisms and translate them into superior drawing 
performance is a fundamental question in understanding artistic expertise, 


one to which we now turn. 


Ways Artists Might See the World Differently 


Consider some hypothetical relations between perception and drawing. If 
artists were born with perceptual systems uniformly superior to those of non- 
artists, they might effortlessly transcribe a percept into a recognizable 
depiction. This non-explanation assumes a categorical distinction between 
artists and non-artists, no variability in either group, and no learning. 
Moreover, it fails to characterize specific mechanisms whereby artists 
outperform non-artists. This mode may be approached by a few drawing 
savants like Nadia, who produced accomplished contour drawings when just 


a few years old (Selfe, 1977), or Stephen Wiltshire, who is capable of 


astonishing levels of iconic visual recall in reproducing aerial views of cities 
in large-scale drawings produced from memory over several days (Treffert, 
2009). However fascinating, such case studies do not inform “normal” skilled 
drawing among competent artists, who typically engage in ongoing 
interactions between perception, cognition, and action (Tchalenko, 2009). 

Alternatively, artists could hypothetically use identical perceptual 
processes as non-artists, possessing no measurable perceptual advantages and 
relying instead on non-perceptual factors like greater motivation in applying 
mundane processes to refine a depiction’s accuracy. In this view, anyone with 
normal vision could learn to draw competently by applying a rote set of 
principles (along these lines, see some art education accounts, e.g. Edwards, 
2012). 

Discounting either extreme view, several more plausible middle-ground 


possibilities remain. For instance, many studies in the classic cognitive 


psychological literature on expertise attribute superior performance in 
domains like chess to the acquisition and organization of thousands of 
domain-specific patterns, or chunks (e.g. Chase & Simon, 1973; Gobet & 
Charness, Chapter 31, this volume). Chunks allow an expert to recognize and 
rapidly encode important features of a situation and to take appropriate 
action. Thus, grand master chess players can detect strategically important 
configurations of chess pieces and use them to reconstruct the positions of 
some two dozen chess pieces after looking at a mid-game board for only a 
few seconds (Chase & Simon, 1973; Gobet & Simon, 1998). However, if the 
pieces are put into meaningless configurations, grand masters’ performances 
plummet, indicating that their apparent advantage in perception and memory 
is relatively fragile, being tied to specific patterns in memory. 

Can a domain-specific, pattern-driven expertise account fully explain 
high-level performance in realistic drawing? While artists, like experts in 
other domains, certainly accumulate significant domain-specific knowledge 


during their training (Gombrich, 1960; Kozbelt & Seeley, 2007), they also 


develop the ability to render even novel objects convincingly. Moreover, as 
discussed below, artists outperform non-artists on perception tasks having 
nothing per se to do with drawing, suggesting a more general perceptual 
advantage (cf. chess). An important difference between visual art and chess is 
that non-chess players need never think about chess, while both artists and 
non-artists routinely need to understand the structure of objects and pictures. 
There is some empirical support for the assertion that artists are superior 
to non-artists in some aspects of visual perception. For instance, Kozbelt 
(2001) compared artists and non-artists on drawing tasks (mostly copying 
line drawings, later judged on accuracy) and higher-order perception tasks 


requiring visual analysis. These included identifying the subjects of blurry 


photos or sets of blobs and lines, and finding a target shape within a more 
complex set of lines (see Figure 30.1). Artists outperformed non-artists on 
both perception and drawing tasks, providing empirical support for the idea 
that artists perceive the world differently (and in some respects better) than 
non-artists. Performance on the two types of tasks was positively correlated; 
Statistically controlling for performance on one or the other type of task 
suggested that artists’ perceptual advantages are best viewed as a subset of 
their drawing skills. In sum, artists’ perceptual advantages are real, and they 


are developed largely to the extent that they are useful in drawing.- 
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Figure 30.1 Kozbelt (2001) demonstrated that expert artists outperformed 
untrained novices in perceptual recognition tasks, such as identifying the 
objects depicted in blurry photos (left) and incomplete line drawings 
(middle), as well as detecting simple shapes embedded in complex 


collections of lines (right). 


Additionally, researchers have addressed other aspects of perception 
associated with observational drawing skill, including superior local — as 
opposed to global — processing (e.g. Chamberlain, McManus, Riley, Rankin, 
& Brunswick, 2013; Drake & Winner, 2011), greater field independence 


(Gaines, 1975), more flexibility in switching between global and local 


attention (Chamberlain & Wagemans, 2015), better integration of local 


details into global representations of objects (Perdreau & Cavanagh, 2014), 


and better memory for changes to to-be-drawn objects and their depictions 


(Perdreau & Cavanagh, 2015). Some of these findings — as well as some 
inconsistent results and relevant methodological issues — are discussed in 
more detail below. 

Overall, initial findings on the ways that the perception of artists is 
superior to that of non-artists are encouraging, but they raise as many 
questions as they answer. What specific cognitive mechanisms facilitate 
depictive skill? Do artists’ advantages extend to low levels of perceptual 
processing? What kind of theoretical account would best characterize the 
nature of artistic expertise? In the next section, we review additional research 
on these issues, organizing it into the two broad theoretical perspectives on 
artists and perception that have dominated research in this area, namely, the 


bottom-up and top-down models. 


Psychological Explanations for Skill in 
Observational Drawing 


Perception involves an interaction between “bottom-up” and “top-down” 
processing. Bottom-up processing is derived exclusively from immediate 
sensory information processed by retinal photoreceptors. Top-down 
processing is influenced by additional cognitive processes beyond the raw 
sensory signal, including endogenous selective attention, explicit knowledge 
about the structure of common objects, and the integration of visual long- 
term memories into a final percept. By analogy, two major sets of 
explanations for drawing ability have been advanced, differentially focusing 
on the importance of bottom-up versus top-down processing to explain 
individual differences in depictive skill (Kozbelt, Seidel, ElBassiouny, Mark, 
& Owen, 2010; Ostrofsky, Kozbelt, & Seidel, 2012). 


Bottom-Up Explanations 


Bottom-up explanations of drawing skill ultimately derive from the 
influential and long-standing idea of the “innocent eye” proposed by art 
historians John Ruskin (1971) and Roger Fry (1960). Couched in modern 
terms, this notion exploits the well-established notion that conscious 
perception of the environment does not perfectly reflect patterns of light 
entering the eye and stimulating retinal photoreceptors. The visual system 
transforms that information, resulting in a percept that functions to infer the 


actual structure of objects and scenes (Purves & Howe, 2005). These 


operations manifest themselves by the near-universally experienced 
phenomena of visual illusions and perceptual constancies. A classic instance 
of this involves the perception of circular objects that project to the retina as 
ellipses; when asked to draw the ellipse or match it to one of a set of ellipses 
of different eccentricities, viewers’ responses are biased toward more circular 
shapes, as if contaminated by the knowledge of the object’s true shape 
(Hammad, Kennedy, Juricevic, & Rajani, 2008; Taylor & Mitchell, 1997; 
Thouless, 1931, 1932). While such non-veridical percepts are adaptive for 


everyday object recognition and visually guided action, the innocent eye 
perspective argues that they interfere with veridical perception, creating 
problems in drawing accurately. In this view, skilled artists draw well 
because they can somehow suppress the influence of such transformations 
and are instead guided predominantly by the veridical two-dimensional 
appearance of the models they are drawing. A similar logic underlies some 
recent accounts of perception in art. For instance, Livingstone, Lafer-Sousa, 


and Conway (2011) found poorer stereopsis among art students compared to 


non-artists and higher rates of strabismus, which compromises stereo vision, 
among accomplished artists, including Rembrandt (Livingstone & Conway, 
2004). Decreased stereopsis would give artists better access to monocular 
depth cues like perspective, shading, and occlusion, which are relevant to 
accurate depiction. In sum, the innocent eye view proposes that the degree to 
which one can perceive the veridical appearance of an object or scene 
determines how accurately one can draw it. 

An emphasis on early perceptual encoding as the primary determinant of 
drawing accuracy also informs psychological research (e.g. Cohen & 
Bennett, 1997) on what may be called the misperception hypothesis of 
drawing errors. This hypothesis has been supported by several studies 
assessing how accurately individuals perceive and draw identical stimuli. For 
instance, Mitchell, Ropar, Ackroyd, and Rajendran (2005) administered 
drawing and perceptual judgment tasks based on the Shepard illusion, 
whereby vertical lines are perceived as longer than horizontal lines when they 
are the same objective length; this illusion is exaggerated if the lines are 
embedded in a depicted three-dimensional object like a table, versus in a two- 
dimensional quadrilateral. Similarly, Ostrofsky, Kozbelt, and Cohen (2015, 
Experiment 2) administered drawing and perceptual judgment tasks 
pertaining to a shape constancy illusion — specifically, that individuals 
perceive angles to be closer to 90 degrees when embedded in a three- 
dimensional object like a cube than in a two-dimensional pattern when the 
angles are the same objective size (see Figure 30.2). Both studies revealed 
congruent patterns of errors in the perceptual judgments and drawing tasks 
and positive correlations between perceptual judgment errors and drawing 


errors. Thus, at least with respect to the perception of relative line length and 


angle size, the degree to which one misperceives a feature of an object 


appears to predict the degree to which one errs in drawing it. 


ae 


Figure 30.2 Shape constancy effect, whereby the size of an angle is 


— 


perceived to be closer to 90 degrees when embedded in a representation of 
a 3D cube than when embedded in a representation of a 2D parallelogram. 
The angles pointed to with an arrow in the top row are both 57 degrees; the 


two angles pointed to in the bottom row are both 136 degrees. 


In light of these findings, an important question for the innocent eye 
theory is whether skilled artists enjoy general advantages in perceptual 
processing in conjunction with their drawing advantages. This has been 
addressed via two methodological strategies: comparing perceptual 
processing of experienced artists versus inexperienced non-artists, and/or 
assessing correlations between general indices of drawing skill and 
perceptual judgment accuracy when the stimuli used in the two tasks are very 
different. In such studies, drawing skill is measured by having participants 
draw from a standardized model (e.g. a photograph of a face, octopus, or 
hand), and drawing accuracy is assessed either by independent raters’ 


subjective judgments or objective measures of differences between the 


drawing and model. Researchers employing one or both of these methods 
have used varied tasks and measures, yielding mixed evidence for the 


“innocent eye.” 


Shape Constancy 


Shape constancy refers to individuals not perceiving the actual shapes of 
objects as changing, even as their apparent shapes change due to different 
viewpoints. This factor creates an illusion whereby the apparent shapes of 
objects are perceived to be closer to their assumed “real” shape than they 
veridically appear, when embedded in a three-dimensional scene, like the 
ellipse example described earlier. Several researchers have assessed the 
relationship between drawing skill or experience and experience of the shape 
constancy illusion (Cohen & Jones, 2008; McManus, Loo, Chamberlain, 
Riley, & Brunswick, 2011; Ostrofsky, Cohen, & Kozbelt, 2014; Ostrofsky et 


al., 2012; Thouless, 1932). Some of these studies have shown negative 


correlations between general drawing skill and the degree to which 


individuals experience shape constancy (Cohen & Jones, 2008; Ostrofsky et 


al., 2014). However, other studies have failed to replicate this finding, using a 
similar shape constancy task but different stimuli (McManus et al., 2011; 


Ostrofsky et al., 2012). Notably, studies focusing on the degree to which 


artists versus non-artists experience the shape constancy effect have generally 
failed to demonstrate reliable group differences (Cohen & Jones, 2008; 
Ostrofsky et al., 2012). Thus, while greater drawing skill appears to be 
associated with a weaker experience of this perceptual illusion in some 
studies, this relationship does not appear to generalize across all types of 


stimuli inducing shape constancy effects. Furthermore, there is not yet sound 


evidence that artistic experience is associated with reduced susceptibility to 


shape constancy. 


Size Constancy 


Size constancy entails individuals not perceiving the physical size of objects 
to change as they move closer or farther away, despite the fact that their 
apparent size does. When two objects of the same apparent size are 
interpreted as being at different distances, viewers experience an illusion 
whereby the “far” object seems larger than the “near” object. Some 
researchers have demonstrated that general measures of drawing skill are 
negatively correlated with the degree to which individuals experience size 
constancy (Ostrofsky et al., 2012, 2014). In such studies, participants 
typically adjust one stimulus to match the size of another, with depth cues 
either present or absent. Ostrofsky et al. (2012) found that both artists and 
non-artists show size constancy effects (i.e. more accurate performance in the 
non-depth condition versus the depth condition) and that artists experience 
size constancy reliably less than inexperienced non-artists do. However, non- 
artists’ baseline performance was far more accurate in the non-depth 
condition than was artists’ performance in the depth condition; thus, artists 
cannot simply override size constancy effects, as implied by a strong version 
of the innocent eye view. Moreover, two other studies, using different tasks 
to measure the illusion, have failed to find any reliable differences between 


artists and non-artists (Chamberlain & Wagemans, 2015; Perdreau & 


Cavanagh, 2011). In sum, evidence on possible relations between greater art 
experience and/or skill in drawing with more accurate size perception 


remains mixed. 


Lightness Constancy and Other Visual Illusions 


Lightness constancy involves individuals not perceiving changes of an 
object’s “real” level of brightness across changes in the luminance conditions 
of the environment. When two patches are equated in apparent brightness, but 
the first is perceived to be under a shadow and the second is not, individuals 
experience an illusion whereby the first patch seems brighter than the second. 


One study comparing artists and non-artists demonstrated that both groups 


experienced this illusion to the same degree (Perdreau & Cavanagh, 2011); to 
our knowledge, no researchers have extended this paradigm to compare 
individuals varying on drawing skill, irrespective of artist versus non-artist 
status. Finally, Chamberlain and Wagemans (2015) compared artists’ and 
non-artists’ experience of the Ebbinghaus illusion (involving relative size 
perception), the Miiller-Lyer illusion (involving relative line length), and the 
Rod-and-Frame illusion (involving line orientation), likewise finding no 
group differences and null correlations between the degree to which 
participants experienced these illusions and general measures of drawing 


accurdcy. 


Summary of Bottom-Up Results 


A strong version of the innocent eye hypothesis is not supported by empirical 
evidence. There has been no strong evidence that skilled or experienced 
drawers are completely able to suppress their experience of visual illusions 
that are produced by perceptual constancy mechanisms. However, a more 
moderate version of this idea, arguing that skill in drawing is associated with 
perceptual processing advantages, has received some empirical support — 


mostly in the form of correlational evidence demonstrating that individuals 


skilled at drawing perceive some visual stimuli more accurately than less 
skilled individuals. This finding has been most consistently demonstrated 
when the visual features that participants draw and perceptually judge are 


identical in type and appearance (Mitchell et al., 2005; Ostrofsky et al., 


2015). When stimuli in perception and drawing tasks differ from one another, 
both in terms of the stimulus appearance and the features being assessed for 
drawing and perceptual judgment accuracy, the evidence is quite mixed: 
some researchers find relations between general drawing and perceptual 
accuracy, and others do not. This conclusion may be due to the use of 
perceptual and drawing tasks that are highly dissimilar. It may also indicate 
that drawing and perceptual processing are not best characterized as a single 
general capacity encompassing all types of visual features. Rather, some 
individuals may be strong in drawing and perceptually processing some 
visual features (e.g. angles), but weaker with respect to others (e.g. brightness 
and size), and the ability to accurately draw and/or perceive different visual 
features may be independent of each another. This idea has been supported 
by evidence like the observation that the magnitudes of size and shape 
constancy errors are not inter-correlated and differentially predict general 
drawing accuracy (Ostrofsky et al., 2012). Thus, the relationship between 
accurately drawing and perceiving visual stimuli may be feature-specific 


rather than generalized (see Ostrofsky et al., 2015). 


Top-Down Explanations 


Besides the limited and conflicting empirical evidence described above, there 
are theoretical arguments as to why a purely bottom-up view is necessarily 
inadequate as an explanation of drawing skill. Critically, bottom-up 
explanations implicitly conceive of drawing as involving passive processing 
of visual information inherent in the model object. In contrast, top-down 
explanations emphasize a wide array of higher-order processes, such as 
explicit domain-specific knowledge, active decision making, and endogenous 
shifts of attention, as major factors inherent in drawing skill (Ostrofsky et al., 
2012). In this view, knowledge-driven influences can facilitate, rather than 
merely interfere with, perception and drawing accuracy. In its pure form, the 
top-down view is thus strongly opposed to the bottom-up view. 

The top-down position is most closely associated with art historian E. H. 
Gombrich (1960; see also Kozbelt & Seeley, 2007). Gombrich argued that 


the inverse optics problem in vision, whereby a retinal image can arise 


through an infinite number of possible configurations of real-world objects, 
applies to depiction as well. When artists render the three-dimensional world 
on a two-dimensional surface, some information must be lost, and other 
information emphasized, to convey the illusion of three-dimensional form 
and space. Artists achieve this goal not by suppressing what they know but 
rather by harnessing their knowledge of the structure of appearances via 
specialized knowledge, or schemata, to meet their depictive goals. Gombrich 
supported his argument with reference to how-to manuals used to train artists 


throughout history (e.g. Cennini, 1954). These manuals and their modern 


counterparts (e.g. Dodson, 1985; Hamm, 1963) supply artists with explicit, 


declarative knowledge of the structure of common object types and means of 
depiction in a given medium, which non-artists lack. Observational drawing 
is not guided exclusively by visual information inherent in the model; rather, 
long-term graphic memories representing the features, proportions, and 
spatial configuration of common objects are activated and partially guide the 


production of an observational drawing. 


Domain-Specific Knowledge of Objects 


Some researchers have found that individual variation in some aspects of 
observational drawings of objects can be predicted by individuals’ drawings 


of those objects produced from imagination (Matthews & Adams, 2008; 


Ostrofsky, 2015). Such studies support the notion that individuals are 
influenced by “what they know” in addition to “what they see” when 
producing observational drawings. Relatedly, Gombrich’s (1960) argument 
about the importance of explicit domain-specific knowledge of objects for 
accurate depiction has been directly supported, with regard to faces. A 
common error in drawing faces involves positioning the eyes too far up the 


head (Ostrofsky et al., 2014). On average, the eye-line is approximately half- 


way down the head (Ostrofsky, 2015). Ostrofsky, Kozbelt, Tumminia, and 


Cipriano (2016) randomly assigned inexperienced non-artists to either 
receive knowledge about proper eye placement or not. Participants receiving 
this knowledge produced smaller errors in vertically positioning the eye-line 
than those who did not, demonstrating that more accurate observational 
drawing is supported by possessing more accurate declarative knowledge of 


the structure of common objects. 


In both laboratory and real-world instructional settings, explicit 
knowledge about object structure is typically communicated via language; 
other forms of skill and knowledge relevant to artistic expertise, perhaps 
especially those involving action and perceptual-motor integration, are harder 
to articulate. As Gombrich (1960), citing Quintilian, noted, “Not everything 
that art can achieve can be passed on” (p. 25). This distinction, between 
aspects of artistic knowledge that are easier versus harder to articulate, limits 
the kinds of artistic knowledge that can be imparted directly via instruction 


and reinforces the multifaceted quality of top-down influences on drawing. 


Decision Making and Efficient Processing of Object Features 


Even so, knowledge of common objects’ structures, without further 
refinement, may be insufficient to produce high-quality drawings. Not all 
features are equally useful for object identification; junctions of lines, for 
instance, are far more important than mid-segments of lines (Biederman, 
1987). Artists may be sensitive to essential features of an object, explicitly or 
intuitively, and use this to decide which aspects to include or emphasize in a 
depiction; non-artists, lacking such sensitivity, would likely make less astute 
decisions. This proposition has been empirically supported in studies using a 
limited-line tracing task, in which participants use a standardized number of 


tape segments to trace over a photo inserted in a transparent sleeve, with the 


goal of producing a recognizable depiction (Kozbelt et al., 2010; Ostrofsky et 
al., 2012). Importantly, participants are not provided with enough tape to 
trace the entire object and thus must decide which features to emphasize. 
Kozbelt et al. (2010) found that artists’ tracings of a face were judged as 


more accurate than those of non-artists, especially when rated by artist 


judges. Ostrofsky and colleagues (2012) replicated this finding using a photo 
of an elephant, and further demonstrated that the judged accuracies of the 
tracings were positively correlated with those of observational pencil 
drawings of an octopus. Moreover, artists included more of several categories 
of line junctions in their tracings of elephants than did non-artists (see Figure 
30.3), a result echoing findings in a free-hand drawing task (Biederman & 
Kim, 2008). In sum, visual selection and decision making — in particular, 
artists’ spontaneous judicious selection of object features important for 
recognition — appear critical for realistic depiction. 
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Figure 30.3 Some images of an elephant produced in the limited-line 
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tracing task reported by Ostrofsky et al. (2012). The top row shows 
tracings produced by trained artists; the bottom row shows tracings 


produced by untrained novices. 


Along similar lines, Perdreau and Cavanagh (2013) argued that artists’ 
perceptual advantages arise from robust representations of object structure in 
memory, which can be used to encode and depict important aspects of objects 


(see also Kozbelt, 2001). Other evidence suggests skilled drawers are more 


efficient, not just more accurate, in perceptual processing of objects, 


compared to unskilled drawers. Perdreau and Cavanagh (2014) had artists 
and non-artists discriminate possible versus impossible figures under various 
exposure durations; drawing skill was operationalized by objective errors in 
copying a photograph of a house. A positive correlation was found between 
the prevalence of drawing errors and average time required to accurately 
discriminate possible versus impossible figures, indicating that individuals 
capable of producing more accurate drawings could more quickly perceive 
the holistic forms of objects. This enhanced efficiency appears to be specific 
to the visual form of objects, as opposed to being generalized across all forms 
of visual processing, since there was no relation between drawing skill and 
time required to accurately discriminate visually presented real versus 
nonsense words in a lexical decision task (cf. results on chess grand masters’ 


perceptual encoding — Chase & Simon, 1973; Gobet & Simon, 1998). 


Visual Attention 


Beyond a possible attentional advantage with respect to selecting what 
features of an object to pay attention to, other researchers have suggested that 
extensive experience in drawing is associated with an ability to shift attention 
between different modes of processing. A high-quality observational drawing 
successfully reproduces both the global structure and local details of the 
model. Researchers have suggested that extensive experience in drawing may 
be associated with a more general advantage in the efficiency of shifting 
attention between the global and local aspects of a stimulus. Chamberlain and 
Wagemans (2015) had participants view stimuli containing squares or circles 
at either the local level (a non-square/circle global shape composed of 


individual square/circle local elements) or the global level (a square/circle 


global shape composed of individual non-square/circle local elements) and 
respond if the display contained a circle or square (regardless of global or 
local level). On any pair of trials, targets could be presented at the same level 
(i.e. at either the local or global level for both trials) or different levels (e.g. at 
the local level for one trial and at the global level for the next trial). In the 
latter case, having to shift attention between global and local levels typically 
slows target identification, compared to consecutive trials at the same level 
(Ward, 1982), presumably due to mutual inhibition between the two levels. 
While both artists and non-artists experienced the expected change in reaction 
time, the cost of switching levels was significantly reduced for artists. This 
may be due to weaker inhibition between processes at the two levels, which 
could support artists’ ability to process both a model’s global structure and 
emerging drawing while also focusing on local details. Weaker inhibition 
between global and local processes may also explain artists’ superior ability 
to integrate isolated local visual information into a global perceptual 


representation (Perdreau & Cavanagh, 2013). 


Summary of Top-Down Results 


There is considerable evidence supporting several aspects of the top-down 
view of artists’ perceptual advantages: explicit knowledge about the structure 
of common objects; greater ability to analyze the structure of objects; and 
enhanced flexibility in shifting attention between different modes of 
perceptual processing all appear to be associated with greater depictive 
ability. Besides such empirical support, top-down advantages do more to 
ground artistic expertise in domain-specific knowledge and specific visual 


abilities than does the bottom-up view. 


Theoretical and Empirical Reconciliations 


So far, we have treated bottom-up and top-down theories as diametrically 
opposed. However, as in real perception, bottom-up and top-down processes 
likely interact and contribute to artistic expertise and drawing skill in varied 
ways. Kozbelt et al. (2010) outlined some ways in which bottom-up and top- 
down views might be conceptually reconciled. For instance, they may apply 
to different depictive problems; bottom-up methods may be best for resolving 
an object’s two-dimensional proportions or clarifying details, while top-down 
methods may facilitate appropriate visual selection. Delineating the meaning 
of knowledge (and its interfering versus facilitating effects) on the two 
accounts is also clarifying; the bottom-up view engages generic knowledge of 
object types useful for everyday perception, while the top-down view regards 
knowledge as highly specialized, artificial, and domain- (or even medium-) 
specific, and useful for understanding object structure and means of 
achieving desired effects in depiction. 

A more integrated understanding might also be had by conceptualizing 
bottom-up and top-down modes of perception as_ strategies, flexibly 
implemented to deal with perceptual ambiguities, rather than mechanistic 
perceptual processes without substantive consideration of context (see 
Ullman, 1984, for such an overtly strategic account of object perception 
involving temporally extended visual “routines”). A more bottom-up strategy 
might involve selecting the most characteristic lines, angles, or shapes upon 
which to construct forms, and assessing overall spatial relationships, for 


instance, in “apprehending the relation of forms and color to one another, as 


they cohere within the object” (Fry, 1960, p. 49). Novices may find bottom- 
up strategies particularly useful, along the lines of some art instruction 
manuals (e.g. Edwards, 2012; Hoffman, 1989). Top-down strategies may 
help resolve perceptual ambiguities based on expectations of a feature at a 
particular location or guide a decision to emphasize a diagnostic feature, 
enhancing recognition of a depicted object (Kozbelt & Seeley, 2007). As 
artists develop an expert knowledge base of declarative patterns and dynamic 
procedures for perception and depiction, top-down schemata (Gombrich, 
1960) likely become increasingly important in guiding perceptual and motor 
processes, facilitating wise selection of viewpoint-dependent information that 
still accurately captures an object’s structure (Kozbelt et al., 2010; Perdreau 
& Cavanagh, 2011, 2014). Finally, experienced artists may have substantial 
strategic flexibility, reverting to more bottom-up strategies when drawing 
unfamiliar objects or correcting depictive errors. 

Further leverage in coordinating bottom-up and top-down views may 
involve reframing the issue away from perception and toward visual attention 
— specifically, the interaction between strategic shifts in attention guiding 
visual selection and the attentional enhancement of selected information and 
suppression of non-selected information (Chamberlain & Wagemans, 2015; 
Kozbelt & Seeley, 2007), as well as the role of attention in visually guided 
action (Seeley & Kozbelt, 2008). As noted earlier, a major problem in 
drawing is the moment-to-moment selection of what to attend to and render. 


Results from limited-line tracing tasks (Kozbelt et al., 2010; Ostrofsky et al., 


2012; see also Tchalenko, 2009) suggest that the selection process differs 
between artists and non-artists and is related to drawing accuracy. While 
bottom-up and top-down strategic attentional shifts reflect the process of 


selecting what information to depict, an additional issue concerns subsequent 


processing of already-selected information. Consider the depth cue condition 
of a typical size matching task (e.g. Ostrofsky et al., 2012). That artists make 
smaller size constancy errors than non-artists may suggest that artists are 
better at focusing attention on task-relevant information — though it is clear to 
both groups what information on a size matching display is relevant. Along 
these lines, artistic skill may involve a capacity to bias attention toward 
enhancing the processing of target information and suppressing task- 
irrelevant information, resulting in a functionally larger pool of attentional 
resources. 

Integrating top-down and bottom-up views can also take a more 
empirical turn, moving beyond studies that have examined only one or the 


other viewpoint (e.g. Cohen & Jones, 2008; Kozbelt et al., 2010), or pitted 


the two against each other (Cohen & Bennett, 1997; Ostrofsky et al., 2012). 


For instance, in one recent investigation (Kozbelt, Snodgrass, & Ostrofsky, 
2014), artists and non-artists created depictions by placing 225 small squares 
of black tape within a 28 x 32 grid superimposed on a photograph of a face. 
Superimposing a standardized grid on the reference image, where each square 
may be either black or white, allows for objective binary coding of each 
“pixel” within the depiction. Moreover, the reference image can be processed 
via image manipulation software to preserve the position, coarseness, and 
size of the grid, and the number of black versus white elements; this can then 
serve as the “best” pixelated drawing, at least in bottom-up terms. An 
analysis comparing square placement in each drawing with that of a 
computer-generated version of the image revealed a large artist advantage in 
sensitivity to placing the squares appropriately — a bottom-up index of 
drawing skill. Subjective accuracy ratings by artist and non-artist judges also 


indicated that artists’ ratings of other artists’ renderings were considerably 


higher than any of the other three drawer—rater combinations (consistent with 
Kozbelt et al., 2010). This method provides a means for further integrating 
bottom-up and top-down explanations of drawing skill that transcends 
defining bottom-up advantages mainly in terms of overcoming perceptual 
constancies, by extending it to include other aspects of the bottom-up signal, 
like relative luminance across an image. It also allows an assessment of how 
participants process the bottom-up signal (in terms of matching the 
distribution of correct answer squares) compared to a top-down, caricatured 
deviation from that signal — in terms of systematic deviations from the “best” 
bottom-up depiction, in the service of greater expressiveness (see Figure 
30.4). 
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Figure 30.4 Some images of a face produced in the pixel drawing task 
reported by Kozbelt et al. (2014). The top row shows tracings produced by 
trained artists; the bottom row shows tracings produced by untrained 


novices. 


Future Directions 


Methodological Issues 


We have identified instances where artistic experience and/or skill in drawing 
appears to be associated with domain-specific and domain-general 
advantages in visual perception, attention, knowledge, and decision making. 
However, significant methodological questions linger in this body of 
research. For instance, virtually all the studies described above have adopted 
correlational or quasi-experimental methods, leaving the direction of 
causality ambiguous. Take findings like smaller size constancy effects among 
experienced artists compared to inexperienced non-artists (Ostrofsky et al., 
2012) or an observed positive correlation between errors produced among 
non-artists in drawing versus perceiving angles (Ostrofsky et al., 2015). Such 
results could indicate that perceptual processing advantages precede and thus 
causally engender drawing advantages, or that developing drawing skill 
causes individuals to perceive more accurately or efficiently, or that they co- 
evolve or co-vary with additional unspecified variables (e.g. motivation). 
Quasi-experimental and correlational designs cannot disentangle these 
possibilities. Thus, an important direction for future research is adopting 
experimental approaches that would, for instance, directly manipulate 
experience in drawing to assess any effects on perception. Longitudinal 
studies, gauging the ongoing time courses of the acquisition of drawing skill 
versus various perceptual abilities, would be especially informative. 

Another major methodological issue, alluded to throughout this chapter, 
involves how expertise in drawing has been operationally defined; either with 
respect to drawing experience, typically through artist versus non-artist 
comparisons, or with respect to drawing skill, via measurement of depictive 


accuracy or errors in standardized drawing tasks. In a few instances, a single 


study has employed both operationalizations and found them to be similarly 
associated with performance on non-drawing tasks — for instance, comparing 
size constancy and visual selection advantages (Ostrofsky et al., 2012), or 
null effects on several visual illusions (Chamberlain & Wagemans, 2015). 
However, experience and skill are not synonymous: Cohen and Jones (2008) 
found that measured drawing skill was negatively correlated with shape 
constancy errors, while simultaneously not finding any difference between 
artists and non-artists in shape constancy errors; Chamberlain and colleagues 
(2013) found a similar pattern with two tasks measuring local perceptual 
processing ability. 

Such conflicting results may be partly attributed to the fact that many 
contemporary modes of art-making do not involve observational drawing. 
Chamberlain et al. (2013) reported that many art students do not self-identify 
as strong observational drawers, and in many studies, “artist” participants are 
college-level art majors or soi-disant professional artists recruited from the 
community with no further specifications. This fact does not guarantee high 
or consistent levels of skill, just as, conversely, “non-artist” status does not 
imply low skill levels. Artist samples are likely to be multifariously 
heterogeneous — in academic or professional level, age, depictive skill level, 
as well as preferred media, style, and type of art produced. Notably, while 
some researchers have reported efforts to ensure high drawing skill among 


their artist participants (e.g. Chamberlain et al., 2013), others have not. More 


formalized, consistent, and transparent screening procedures should be 
adopted to resolve these issues. Additional selection criteria, involving, say, 
sub-groups of artists with different aesthetic judgment criteria, could also be 
informative about both producing and judging depictions (Serafin, Kozbelt, 


Seidel, & Dolese, 2011). Along these lines, a useful test case of the nature, 


scope, and source of potential perceptual advantages associated with 
expertise would involve extending existing testing paradigms to samples of 
visually engaged art experts, such as historians, curators, or critics, who 
themselves do not draw: would they show similar perceptual trends as trained 
artists, compared to non-artists? 

Just as researchers have used two putatively associated operational 
definitions of drawing expertise, they have also implemented multiple 
dependent measures of drawing accuracy. The two main approaches are 
subjective accuracy ratings by independent judges (artists or non-artists) and 
objective metrics of drawing accuracy — each, alas, with some limitations. 
Ratings are arguably limited in their inherent subjectivity, though achieving 
high inter-rater reliability alleviates this concern somewhat (Amabile, 1982). 
Moreover, most drawing studies treat subjective accuracy ratings holistically 
— that is, representing a unidimensional accuracy continuum. However, this is 
a limited conceptualization. A drawing may show high accuracy in some 
respects (e.g. the relative spatial proportions of different features of a model), 
but low accuracy in other ways (e.g. inconsistent brightness gradients across 
the depicted object or unconvincing articulation of parts into a coherent 
whole). A few researchers have attempted to improve upon this by having 
judges provide separate accuracy ratings for different aspects of drawings. 
Cohen and Earls (2010) and Cohen and Jones (2008) had judges give 
separate ratings for isolated facial features and the spatial relationships 
between the features — but this remains unusual. 

Holistic ratings also do not inform which aspects of a depiction 
contribute to an overall sense of realism. It is often assumed that artistic 
realism is self-evident. As Leonardo da Vinci noted, “A painter should not 


object to listening to the opinion of a layman. Even if a man is not a painter, 


he knows what the human form looks like ... If someone is quite capable of 
judging the works of Nature, shouldn’t we admit him capable of detecting our 
errors?” (Kelen, 1990, p. 139). However, realism remains a surprisingly 
slippery construct, and artists and non-artists sometimes differ significantly in 
their subjective accuracy ratings (Kozbelt et al., 2010, 2014), suggesting that 
the two groups may use different criteria or standards (see also Kaufman, 
Baer, Cole, & Sexton, 2008; Runco, McCarthy, & Svenson, 1994; Serafin et 
al., 2011). 


The limitations of subjective ratings have incited the development of 


drawing accuracy measures that assess errors via objectively measurable 
differences between the drawing and model (e.g. Carson & Allard, 2013; 
Ostrofsky et al., 2015). Such methods have advanced researchers’ ability to 
operationalize depictive accuracy with respect to proportions and the relative 
Spatial positioning of a model’s features, but they leave unaddressed other 
aspects of accuracy, like luminance gradients and the appearance of local 
features. Future research may benefit from adopting an integrated approach to 
assessing drawing accuracy using both subjective and objective measures, as 
in some recent studies (Chamberlain, McManus, Riley, Rankin, & 
Brunswick, 2014; Hayes & Milne, 2011; Kozbelt et al., 2014; Ostrofsky et 
al., 2014; Perdreau & Cavanaugh, 2014). 


Considerable near-term progress in understanding artistic expertise 


could be made simply by addressing many of these methodological issues 
head-on — especially given the frequency of conflicting results on 
fundamental questions, like the relation of shape constancy to drawing 
accuracy. The basic replication, extension, and integration of findings, 
ultimately with a view to meta-analyses, would contribute greatly toward 


resolving many outstanding issues. 


Conceptual Issues 


Besides future directions stemming from methodological refinements, some 
broader conceptual issues also open promising avenues for exploration. For 
instance, in the context of the classic expertise literature, an issue raised near 
the outset of this chapter — determining the extent to which depictive skill 
may entail robust, general advantages in perception and attention — remains 
highly pertinent: indeed, a guiding principle in broad conceptualizations of 
the nature of artistic expertise. Much of the evidence to date suggests that 
artists do enjoy fairly generalizable advantages in some aspects of perception, 
perhaps most clearly in understanding object structure (e.g. Kozbelt et al., 
2010; Perdreau & Cavanagh, 2014). Following the logic of Gombrich (1960), 


such an advantage can be attributed to artists routinely needing to solve the 


same kinds of problems in making depictions as the visual system does 
generally for understanding the world. The dynamic described throughout 
this chapter is consistent with Ericsson and Lehman’s (1996) characterization 
of skill acquisition involving the adaptation of pre-existing mechanisms (in 
this case basic perceptual processes) to particular task constraints (making 
visually accurate two-dimensional artistic depictions based on observation of 
the three-dimensional world). 

Characterizing in detail how artists engage particular perceptual and 
attentional mechanisms and translate them into superior drawing performance 
remains a challenge. For instance, how deeply into the visual system do 
perceptual differences between artists and non-artists run? To date, research 
on relatively low levels of perceptual processing — such as psychophysical 


indices of perceptual organization thresholds among artists and non-artists on 


several Gestalt grouping principles (Ostrofsky, Kozbelt, & Kurylo, 2013) — 


has yielded null findings. Along similar lines, Perdreau and Cavanagh (2011) 
framed their investigation as “Do artists see their retinas?” and argued for an 
answer in the negative. Furthermore, Cohen and Bennett (1997) 
provocatively hypothesized a distinction between illusions, which are rooted 
in low-level, cognitively impenetrable mechanisms, and delusions due to the 
interfering effect of knowledge, which they argued are responsible for most 
drawing errors but which can in principle be overcome. 

Finding meaningful artistic expertise-based psychophysical differences 
at very low levels of visual processing — such as visual acuity, color 
perception, or contrast sensitivity — may be daunting, but there is a precedent 
in the study of first-person action video game playing. In a noteworthy series 
of studies, Bevalier and colleagues (e.g. Li, Polat, Makous, & Bevalier, 2009) 
found systematic changes in low-level visual processing, such as enhanced 
contrast sensitivity, as a result of video game playing — even when conditions 
were randomly assigned. Such results inform ongoing general debates about 


the mutual influences of experience, cognition, and perception, at various 


levels of processing (see Firestone & Scholl, 2015; Landy, Chapter 10, this 
volume). Thus, it may yet be possible to find parallel deep changes in the 
perception of artists as they intensely engage the visual world with the goal of 
depicting it, though no direct evidence for this has yet been found (e.g. 
Perdreau & Cavanagh, 2011). 

Additionally, understanding motoric aspects of artistic expertise, and 
their interaction with perception, is another issue awaiting in-depth 
exploration. Motor processes have been extensively studied in dance, sports, 
and other perceptual-motor-intensive domains (e.g. Rosenbaum, Augustyn, 


Cohen, & Jax, 2006; Williams, Ford, Hodges, & Ward, Chapter 34, this 


volume). Attempts to isolate fine motor differences as a function of artistic 
expertise have generally yielded null results (e.g. Cohen & Bennett, 1997); 
however, hand and eye movements are known to have strong bidirectional 
influences, so examining their mutual interplay may be necessary (Gowen & 
Miall, 2006). Consistent with other findings on expertise, recent work 
examining eye and hand movements in naturalistic drawing has found that 
artists can produce more motor output per unit of visually encoded material 


when drawing, relative to non-artists (Glazek, 2012); moreover, artists use a 


systematic eye—hand strategy while segmenting complex lines, while non- 


artists either segment arbitrarily or not at all (Tchalenko, 2009). Perceptual- 


motor integration themes feature prominently in many recent accounts of 
depiction (Kozbelt, 2001; Kozbelt & Seeley, 2007; Seeley & Kozbelt, 2008; 
Tchalenko, Nam, Ladanga, & Miall, 2014) and, together with the theme of 
embodied cognition (e.g. Lakoff & Johnson, 1999), are poised to become 


increasingly important in characterizing artistic expertise, as are explorations 
of individual-difference variables like personality and motivational factors 
(see Chamberlain, McManus, Brunswick, Rankin, & Riley, 2015). 

Besides laboratory studies comparing artists and non-artists that have 
comprised the bulk of this chapter, other approaches, which can only be 
touched on briefly here, are also promising. One is a continued focus on 
special populations besides professional artists, broadly construed. For 
example, contemporary drawing prodigies represent a rich source of data on 
specific aspects of perceptual processing that undergird realistic depiction 


ability (Drake & Winner, 2012; Winner, 1996). Such investigations can be 


supplemented by archival studies of great artists bearing on questions of 
expertise development, as in the childhood work of Klee, Toulouse-Lautrec, 


or Picasso (Pariser, 1991), and contemporary case study accounts of 


accomplished artists, as in Solso’s (2001) neuroimaging study of portraitist 
Humphrey Ocean. 

Finally, the details of artists’ domain-specific knowledge and schemata 
(Gombrich, 1960; Kozbelt & Seeley, 2007) themselves remain woefully 
under-characterized. For instance, the extent to which domain-specific 
knowledge and perceptual skills might be modulated across artistic media is 
an intriguing open question. Consider the task requirements of different 
media: the fine linear detail of drypoint etching versus large-scale perception 
and motor execution in mural painting versus three-dimensional spatial 
reasoning in sculpture: is expertise in such varied media associated with 
different constellations of perceptual strengths? Characterizing artists’ 
knowledge base and its relation to specific perceptual and motor skills would 
also inform issues in the appreciation and connoisseurship of art (Bullot & 


Reber, 2013; Kozbelt & Ostrofsky, 2013), art education and pedagogy 


(Edwards, 2012), as well as better understanding how artistic expertise and 


skill facilitate creativity (Galenson, 2001; Kozbelt, 2008). Understanding 


artists’ skills and knowledge is also relevant to linking the expertise literature 
with accounts of how the supposed basic mechanisms that undergird human 
artistry emerged and/or were co-opted in our evolutionary past (Tumer, 
2006). 


Conclusion 


Artists are situated at the intersection of a complex set of issues within the 
study of expertise, including visual perception and cognition, motor 
processes, historical and socio-cultural factors, and creative thought. The 
study of artistic expertise — past, present, and hopefully future — touches on a 
broad spectrum of issues within psychology and allied domains. The 
complexity and inherently interdisciplinary quality of visual art may be partly 
responsible for the seemingly late start of a significant research tradition on 
expertise in the domain of art, and, even at this date, the relative rarity of 
empirical studies on expertise in drawing. 

However, the fact that researchers on artistic expertise have had to play 
catch-up to other, better-established, domains of expertise is not completely 
disadvantageous. The ongoing development of models of expert performance 
in other domains provide means to gauge the extent to which — and precisely 
how — the domain of visual art may (or may not) differ from other domains. 
That visual artists creating depictions routinely need to solve many of the 
same problems as the visual system does generally suggests that some aspects 
of artists’ perceptual advantages may transcend specific learned patterns and 
offer greater flexibility in visual analysis. To date, there are a number of 
suggestive findings, on the relation between certain aspects of early visual 
processing and the accuracy of depictions of isomorphic stimuli, on the 
importance of visual selection, and on likely artist advantages in 
understanding object structure, but little yet in the way of incontrovertible 


facts. Simply fleshing out the many lines of nascent or potential inquiry 


described above could be very informative, facilitating the testing of 
predictions about high-level performance in art-making based on empirical 
findings about experts in other domains. Reciprocally, the potentially deep 
influence of artists’ expertise on their perceptual capabilities could 


conceivably re-characterize how expertise itself is typically understood. 


References 


Amabile, T. M. (1982). Social psychology of creativity: A consensual 
assessment technique. Journal of Personality and Social Psychology, 43, 
997-1013. 


Biederman, I. (1987). Recognition-by-components: A theory of human image 


understanding. Psychological Review, 94, 115-117. 


Biederman, I., & Kim, J. G. (2008). 17,000 years of depicting the junction of 
two smooth shapes. Perception, 37, 161-164. 


Brown, D. (1991). Human universals. New York: McGraw-Hill. 


Bullot, N. J., & Reber, R. (2013). The artful mind meets art history: Toward a 
psycho-historical framework for the science of art appreciation. Behavioral 
and Brain Sciences, 36, 123—137. 


Carson, L., & Allard, F. (2013). Angle-drawing accuracy as an objective 
performance-based measure of drawing expertise. Psychology of Aesthetics, 
Creativity, and the Arts, 7, 119-129. 


Cennini, C. d’A. (1954). The craftsman’s handbook (II libro dell’arte) (trans. 
D. V. Thompson, Jr.). New York: Dover. (Original work published early 
fifteenth century) 


Chamberlain, R., McManus, I. C., Brunswick, N., Rankin, Q., & Riley, H. 


(2015). Scratching the surface: Practice, personality, approaches to learning, 


and the acquisition of high-level representational drawing ability. Psychology 
of Aesthetics, Creativity, and the Arts, 9, 451-462. 


Chamberlain, R., McManus, I. C., Riley, H., Rankin, Q., & Brunswick, N. 
(2013). Local processing enhancements associated with superior 
observational drawing are due to enhanced perceptual functioning, not weak 
central coherence. Quarterly Journal of Experimental Psychology, 66, 
1448-1466. 


Chamberlain, R., McManus, I. C., Riley, H., Rankin, Q., & Brunswick, N. 
(2014). Cain’s house task revisited and revived: Extending theory and 
methodology for quantifying drawing accuracy. Psychology of Aesthetics, 
Creativity, and the Arts, 8, 152-167. 


Chamberlain, R., & Wagemans, J. (2015). Visual arts training is linked to 
flexible attention to local and global levels of visual stimuli. Acta 
Psychologica, 161, 185-197. 


Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive 
Psychology, 4, 55-81. 


Cohen, D. J., & Bennett, S. (1997). Why can’t most people draw what they 
see? Journal of Experimental Psychology: Human Perception and 
Performance, 23, 609-621. 


Cohen, D. J., & Earls, H. (2010). Inverting an image does not improve 
drawing accuracy. Psychology of Aesthetics, Creativity, and the Arts, 4, 
168-172. 


Cohen, D. J., & Jones, H. E. (2008). How shape constancy relates to drawing 


accuracy. Psychology of Aesthetics, Creativity, and the Arts, 2, 8-19. 


Cupchik, G. C. (1992). From perception to production: A multilevel analysis 
of the aesthetic process. In G. C. Cupchik & J. Laszlo (eds.), Emerging 
visions of the aesthetic process (pp. 83-99). Cambridge University Press. 


Dodson, B. (1985). Keys to drawing. Cincinnati, OH: North Light Books. 


Drake, J. E., & Winner, E. (2011). Realistic drawing talent in typical adults is 
associated with the same kind of local processing bias found in individuals 
with ASD. Journal of Autism and Developmental Disorders, 41, 1192-1201. 


Drake, J., & Winner, E. (2012). Predicting artistic brilliance. Scientific 
American Mind, 23, 42—48. 


Edwards, B. (2012). Drawing on the right side of the brain. New York: 


Penguin. 


Ericsson, K. A., & Lehmann A. C. (1996). Expert and exceptional 
performance: Evidence of maximal adaptation to task constraints. Annual 
Review of Psychology, 47, 273-305. 


Feist, G. J. (2004). The evolved fluid specificity of human creative talent. In 
R. J. Sternberg, E. L. Grigorenko, & J. L. Singer (eds.), Creativity: From 
potential to realization (pp. 57-82). Washington, DC: American 


Psychological Association. 


Firestone, C., & Scholl, B. J. (2015). Cognition does not affect perception: 
Evaluating the evidence for ‘top-down’ effects. Behavioral and Brain 
Sciences, 20, 1—77. 


Fry, R. (1960). Vision and design. New York: Meridian Books. (Original 
work published 1919) 


Gaines, R. (1975). Developmental perception and cognitive styles: From 
young children to master artists. Perceptual and Motor Skills, 40, 983-998. 


Galenson, D. W. (2001). Painting outside the lines: Patterns of creativity in 


modern art. Cambridge, MA: Harvard University Press. 
Gardner, H. (1983). Frames of mind. New York: Basic Books. 


Glazek, K. (2012). Visual and motor processing in visual artists: Implications 
for cognitive and neural mechanisms. Psychology of Aesthetics, Creativity, 
and the Arts, 6, 155-167. 


Gobet, F., & Simon, H. A. (1998). Expert chess memory: Revisiting the 
chunking hypothesis. Memory, 6, 225-255. 


Goldwater, R., & Treves, M. (eds.) (1972). Artists on art from the 14th to the 
20th century. New York: Pantheon. 


Gombrich, E. H. (1960). Art and illusion. Princeton University Press. 


Gowen, E., & Miall, R. C. (2006). Eye—hand interactions in tracing and 
drawing tasks. Human Movement Science, 25, 568-585. 


Hamm, J. (1963). Drawing the head and figure. New York: Grosset & 
Dunlap. 


Hammad, S., Kennedy, J. M., Juricevic, I., & Rajani, S. (2008). Ellipses on 
the surface of a picture. Perception, 37, 504—510. 


Hayes, S., & Milne, N. (2011). What’s wrong with this picture? An 
experiment in quantifying accuracy in 2D portrait drawing. Visual 
Communication, 10, 149-174. 


Hoffman, H. S. (1989). Vision and the art of drawing. Englewood Cliffs, NJ: 


Prentice-Hall. 


Kaufman, J. C., Baer, J., Cole, J. C., & Sexton, J. D. (2008). A comparison of 
expert and nonexpert raters using the consensual assessment technique. 
Creativity Research Journal, 20, 171-178. 


Kelen, E. (ed.) (1990). Leonardo da Vinci’s advice to artists. Philadelphia: 


Running Press. 


Kozbelt, A. (2001). Artists as experts in visual cognition. Visual Cognition, 8, 
705-723. 


Kozbelt, A. (2008). Hierarchical linear modeling of creative artists’ problem 


solving behaviors. Journal of Creative Behavior, 42, 181—200. 


Kozbelt, A., & Ostrofsky, J. (2013). Extending the psycho-historical 
framework to understand artistic production. Behavioral and Brain Sciences, 
36, 148-149. 


Kozbelt, A., & Seeley, W. P. (2007). Integrating art historical, psychological, 
and neuroscientific explanations of artists’ advantages in drawing and 


perception. Psychology of Aesthetics, Creativity, and the Arts, 1, 80-90. 


Kozbelt, A., Seidel, A., ElBassiouny, A., Mark, Y., & Owen, D. R. (2010). 


Visual selection contributes to artists’ advantages in realistic drawing. 


Psychology of Aesthetics, Creativity, and the Arts, 4, 93-102. 


Kozbelt, A., Snodgrass, E., & Ostrofsky, J. (2014). Pixel drawing: A novel 
signal detection-based approach to measuring drawing skill. In A. Kozbelt 
(ed.), Proceedings of the Twenty-Third Biennial Congress of the 
International Association of Empirical Aesthetics (pp. 276-281). New York: 
IAEA. 


Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied 


mind and its challenge to western thought. New York: Basic Books 


Li, R., Polat, U., Makous, W., & Bevalier, D. (2009). Enhancing the contrast 
sensitivity function through action video game training. Nature 
Neuroscience, 12, 549-551. 


Livingstone, M. S., & Conway, B. R. (2004). Was Rembrandt stereoblind? 
New England Journal of Medicine, 351, 1264-1265. 


Livingstone, M. S., Lafer-Sousa, R., & Conway, B. R. (2011). Stereopsis and 
artistic talent: Poor stereopsis among art students and established artists. 
Psychological Science, 22, 336-338. 


Martindale, C. (1990). The clockwork muse: The predictability of artistic 
change. New York: Basic Books. 


Matthews, W. J., & Adams, A. (2008). Another reason why adults find it 
hard to draw accurately. Perception, 37, 628-630. 


McManus, I. C., Loo, P.-W., Chamberlain, R. C., Riley, H. & Brunswick, N. 
(2011). Does shape constancy relate to drawing ability? Two failures to 
replicate. Empirical Studies of the Arts, 29, 191-208. 


Melcher, D., & Cavanagh, P. (2011). Pictorial cues in art and in visual 
perception. In F. Bacci & D. Melcher (eds.), Art and the senses 
(pp. 359-394). Oxford University Press. 


Mitchell, P., Ropar, D., Ackroyd, K., & Rajendran, G. (2005). How 
perception impacts on drawings. Journal of Experimental Psychology: 
Human Perception and Performance, 31, 996-1003. 


Murray, C. (2003). Human accomplishment. New York: Basic Books. 


Ostrofsky, J. (2015). Do graphic long-term memories influence the 
production of observational drawings? The relationship between memory- 
and observation-based face drawings. Psychology of Aesthetics, Creativity, 
and the Arts, 9, 217-227. 


Ostrofsky, J., Cohen, D. J., & Kozbelt, A. (2014). Objective versus subjective 
measures of face-drawing accuracy and their relations with perceptual 
constancies. Psychology of Aesthetics, Creativity, and the Arts, 8, 486-497. 


Ostrofsky, J., Kozbelt, A., & Cohen, D. J. (2015). Observational drawing 
biases are predicted by biases in perception: Empirical support of the 
misperception hypothesis of drawing accuracy with respect to two angle 


illusions. Quarterly Journal of Experimental Psychology, 68, 1007-1025. 


Ostrofsky, J., Kozbelt, A., & Kurylo, D. (2013). Perceptual grouping in 
artists and non-artists: A psychophysical comparison. Empirical Studies of 
the Arts, 31, 131-143. 


Ostrofsky, J., Kozbelt, A., & Seidel, A. (2012). Perceptual constancies and 


visual selection as predictors of realistic drawing skill. Psychology of 


Aesthetics, Creativity, and the Arts, 6, 124-136. 


Ostrofsky, J., Kozbelt, A., Tumminia, M., & Cipriano, M. (2016). Why do 
non-artists draw the eyes too far up the head? How vertical eye-drawing 
errors relate to schematic knowledge, pseudoneglect and context-based 
perceptual biases. Psychology of Aesthetics, Creativity, and the Arts, 10, 
332-343. 


Palmer, S. E. (1999). Vision science: Photons to phenomenology. Cambridge, 
MA: MIT Press. 


Pariser, D. (1991). Normal and unusual aspects of juvenile artistic 
development in Klee, Lautrec, and Picasso. Creativity Research Journal, 4, 
51-65. 


Perdreau, F., & Cavanagh, P. (2011). Do artists see their retinas? Frontiers in 


Human Neuroscience, 5, 1-10. 


Perdreau, F., & Cavanaugh, P. (2013). The artist’s advantage: Better 
integration of object information across eye movements. i-Perception, 4, 
380-395. 


Perdreau, F., & Cavanaugh, P. (2014). Drawing skill is related to the 


efficiency of encoding object structure. i-Perception, 5, 101-119. 


Perdreau, F., & Cavanagh, P. (2015). Drawing experts have better visual 
memory while drawing. Journal of Vision, 15, 5. DOI: 10.1167/15.5.5. 


Pfeiffer, J. E. (1982). The creative explosion. New York: Harper & Row. 


Purves, D., & Howe, C. Q. (2005). Perceiving geometry: Geometrical 


illusions explained by natural scene statistics. New York: Springer. 


Rosenbaum, D. A., Augustyn, J. S., Cohen, R. G., & Jax, S. A. (2006). 
Perceptual-motor expertise. In K. A. Ericsson, N. Charness, R. R. Hoffman, 
& P. J. Feltovich (eds.), The Cambridge handbook of expertise and expert 
performance (pp. 505-520). Cambridge University Press. 


Runco, M. A., McCarthy, K. A., & Svenson, E. (1994). Judgments of the 
creativity of artwork from students and professional artists. Journal of 
Psychology, 128, 23-31. 


Ruskin, J. (1971). The elements of drawing. New York: Dover. (Original 
work published 1857) 


Schlewitt-Haynes, L. D., Earthman, M. S., & Burns, B. (2002). Seeing the 
world differently: An analysis of descriptions of visual experiences provided 


by visual artists and nonartists. Creativity Research Journal, 14, 361-372. 


Seeley, W. P., & Kozbelt, A. (2008). Art, artists, and perception: A model for 
premotor contributions to perceptual analysis and form recognition. 
Philosophical Psychology, 21, 149-171. 


Selfe, L. (1977). Nadia: A case of extraordinary drawing ability in an autistic 


child. London: Academic Press. 


Serafin, J., Kozbelt, A., Seidel, A., & Dolese, M. (2011). Dynamic evaluation 
of high- and low-creativity drawings by artist and non-artist raters: 
Replication and methodological extension. Psychology of Aesthetics, 
Creativity, and the Arts, 5, 350-359. 


Solso, R. L. (2001). Brain activities in an expert versus a novice artist: An 


fMRI study. Leonardo, 34, 31-34. 


Taylor, L. M., & Mitchell, P. (1997). Judgments of apparent shape 
contaminated by knowledge of reality: Viewing circles obliquely. British 
Journal of Psychology, 88, 653-670. 


Tchalenko, J. (2009). Segmentation and accuracy in copying and drawing: 
Experts and beginners. Vision Research, 49, 791-800. 


Tchalenko, J., Nam, S.-H., Ladanga, M., & Miall, R. C. (2014). The gaze- 
shift strategy in drawing. Psychology of Aesthetics, Creativity, and the Arts, 
8, 330-339. 


Thouless, R. H. (1931). Phenomenal regression to the real object. British 
Journal of Psychology, 21, 339-359. 


Thouless, R. H. (1932). Individual differences in phenomenal regression. 
British Journal of Psychology, 22, 216-241. 


Treffert, D. A. (2009). Savant syndrome: An extraordinary condition. A 
synopsis: Past, present, future. Philosophical Transactions of the Royal 
Society of London B: Biological Sciences, 364, 1351-1357. 


Tumer, M. (ed.) (2006). The artful mind: Cognitive science and the riddle of 


human creativity. Oxford University Press. 
Ullman, S. (1984). Visual routines. Cognition, 18, 97-159. 


Vasari, G. (1996). Lives of the painters, sculptors, and architects, 2 vols. 
(trans. G. du C. de Vere). New York: Knopf. (Original work published 1550) 


Ward, L. M. (1982). Determinants of attention to local and global features of 
visual forms. Journal of Experimental Psychology: Human Perception and 
Performance, 8, 562-581. 


Willats, J. (1997). Art and representation: New principles in the analysis of 


pictures. Princeton University Press. 


Winner, E. (1996). The rage to master: The decisive case for talent in the 
visual arts. In K. A. Ericsson (ed.), The road to excellence: The acquisition of 
expert performance in the arts and sciences, sports and games (pp. 271-301). 
Mahwah, NJ: Erlbaum. 


1 In passing, note that while such a claim is useful in demarcating an initial 
theoretical orientation for understanding artists’ perception, it provides few 
details about specific processes and mechanisms, as well as the extent to 
which such advantages potentially transcend standard domain-specific, 
pattern-matching accounts of expertise. 


31 


Expertise in Chess 


& 


Fernand Gobet 
Department of Psychological Sciences, University of Liverpool, UK 
Neil Charness 
Department of Psychology, Florida State University, Florida 


Expertise in Chess 


Chess has long served as a model task environment for research into 
psychological processes (Chamess, 1992). Some of the earliest systematic 
work on individual differences in imagery (Binet, 1966), memory (Djakow, 
Petrowski, & Rudik, 1927), and problem-solving (de Groot, 1965) took place 


in the domain of chess. Of all domains of expertise that have been researched, 


chess is probably the one that has had the most impact, not only for research 
into expertise, but also for psychology in general. 

De Groot (1965) ushered in the modern era of investigation using small 
groups of expert and grand master level players in experimental studies. It 
was his finding of the dissociation between thinking skills and perceptual- 
memory skills that laid the groundwork for subsequent research. When 
asking players to think aloud while they attempted to choose the best move in 
an unfamiliar position, de Groot discovered that, contrary to popular lore, the 
most proficient players did not think further ahead than less skilled players. 
In contrast, memory for briefly presented (2-15 s) chess positions markedly 
differentiated skill levels. De Groot interpreted these findings to support the 
importance of knowledge and perceptual organization principles over search 
algorithm differences in explaining how experts chose better moves. In 
follow-up research, Chase and Simon (1973a, 1973b) examined the 
perceptual and memory mechanisms underpinning skilled performance and 
expert performance in chess and developed their influential chunking theory. 


That theory and its subsequent refinement has had a significant impact on 


expertise research in general (Gobet, 2016) and that on board games in 
particular (see Gobet, de Voogt, & Retschitzki, 2004). 


Brief Description of the Game and the Rating 
System 


Chess is a game played by two opponents from the starting position: an array 
of 32 chess pieces placed on an 8 x 8 square chessboard. The rules of chess 
are sufficiently simple that children can be taught them at a very young age (4 
or 5 years old). Child prodigies are not uncommon and teenagers increasingly 
compete at the highest level. Chess is sufficiently difficult to play well that it 
took computer scientists 40 years of effort to program computers to compete 
successfully with the best human players. 

Another important feature of chess is the existence of a sophisticated 


measurement scale for evaluating chess skill based on performance in 


tournaments. The Elo rating scale (Elo, 1986), available since the mid-1960s, 
is open-ended, starting at a nominal value of zero and extending upward, with 
a nominal class interval (standard deviation) of about 200 rating points. The 
world’s best players today hover above 2,800 rating points. Grand masters 
have approximately 2,500 rating points, international masters about 2,400 
points, and masters about 2,200 points. This interval level rating scale 
enables fine-grained examination of the relation between expertise and a 
variety of indicators of psychological processes. For instance, a psychometric 


approach to chess skill (e.g. Van der Maas & Wagenmakers, 2005) can 


capitalize on the chess rating scale to examine how well it correlates with 
different markers of psychological processes such as measures of memory, 


problem-solving, and motivation. 


In this chapter, we focus on a process model approach to understanding 
expertise in chess. Our goal is to shed light on the process of how players 
choose the best moves to play in a chess game, starting from early perceptual 
processes and tracing forward to the search processes first described by de 
Groot. We outline where skill differences arise within such processes. We 
describe computer simulation models that capture some of the features of 
skilled performance, how human players acquire the knowledge necessary to 
play chess expertly, and discuss other individual differences such as gender, 
intelligence, and personality. (Data on the neuroscience of chess are 
discussed in Bilali¢ & Campitelli, Chapter 14, this volume, and data on aging 
and chess are discussed in Krampe & Charness, Chapter 42, this volume.) 


Information Processing Models of Choosing a 
Good Move: The Trade-Offs between 
Knowledge and Search 


The goal of a chess player is to choose the best possible move. Often, when 
playing through standard openings, the best move is dictated by knowledge 
from published analyses. Chassy and Gobet (2011) estimated that masters 
have memorized at least 100,000 opening moves. Sometimes, detecting the 
best move in a sequence of exchanges of pieces is simple enough such that 
even novices quickly find the solution. Much of the time, the best move is not 
obvious and the player must decide based on a search process that evaluates a 
candidate move in terms of potential future positions reached via a branching 
tree of available moves for the two sides. 

Search is difficult because of the enormous number of possible moves 
stemming from the opening position. Even master players who can use well- 
tuned recognition processes to prune the possible base moves down to three 
or four plausible alternatives at each branch in the tree face a dilemma. The 
number of possible moves, computed as breadth raised to a power equal to 
depth, is 4”° moves, given that the average master game lasts about 38 moves 
or 76 plies (moves for each side). So, both computers and humans must 
search selectively among the alternatives using a variety of heuristics (Newell 


& Simon, 1972) to decide when a node reached in search can be properly 


evaluated. 
Given the relatively slow rate at which moderately skilled players can 


generate analysis moves, about four moves per minute (Charness, 1981b), it 


is obvious that much of the time that human players spend is not in 
generating all possible moves (perhaps taking a move per second) but in 
generating moves selectively and using complex evaluation functions to 
assess their value. By contrast, computer chess programs can achieve high- 
level play by searching many moves using fast, frugal evaluation processes 
that involve minimal chess knowledge to evaluate the terminal positions in 
search. How humans can do so little search, yet still manage to find strong 
chess moves, has focused attention on understanding the role of pattern 
recognition processes in move selection. As de Groot noted, skilled players 
use their knowledge about chess configurations to generate plausible moves 


for limited searching. 


Tracing Expertise Differences in Perception and Attention with Eye- 
Tracking Techniques 

Jongman (1968) initiated work on perceptual skill differences by examining 
eye movements of expert and less expert players, further elaborated in re- 
analysis by de Groot and Gobet (1996). In a memorizing task, where players 
were given a few seconds to examine an unfamiliar chess position, better 
players fixated more on the edges of squares than weaker players did. Also, 
better players were more likely to have greater distances between fixations, 
implying that they were able to encode more widely about a fixation than 
weaker players. Experts also made shorter duration fixations than did weaker 
players implying faster encoding. 

Reingold and colleagues confirmed the larger visual span for experts 
using a variety of tasks. Using a gaze-contingent paradigm that manipulated 
the number of visible squares, Reingold, Charness, Pomplun, and Stampe 


(2001a) showed that skilled players needed a larger area around fixation to 


detect changes in successively displayed chess positions in order to match 
performance under unlimited view of the whole board when compared to less 
skilled players. This was only true for structured, not random chess positions. 
This result suggested that better players had a larger visual field from which 
they could extract chess relationships. In a second experiment, the authors 
noted that when players made a simple determination of whether a King was 
in check by an attacking piece on a minimized chessboard (3 x 3 squares), 
experts required fewer fixations to decide and these fixations were more 
likely to be between pieces (on empty squares), compared to intermediate- 


level players. 


In a choose-a-move task using full chessboards, Charness, Reingold, 
Pomplun, and Stampe (2001) demonstrated that experts made fewer fixations 
per trial and those fixations were more widely spaced out across the board 
and again, more likely to be between than on chess pieces. More importantly, 
for the first five fixations, experts were more likely to fixate on relevant 
squares (rated as relevant by a strong international master player). This very 
early advantage (within the first second of exposure to a new position, given 
that fixations average about 250 ms each for both experts and intermediate 
players) testifies to the importance of pattern recognition processes in 
providing a better representational structure. Given this perceptual head start, 
experts also chose better moves and did so more quickly than their less expert 
counterparts. 

Reingold, Charness, Schultetus, and Stampe (2001b) used a Stroop-like 
interference task within a 5 x 5 square segment of a chessboard to 
demonstrate that expert players appear to extract chess relations in parallel, 
whereas weaker ones appear to shift attention and encode the same relations 
serially. In a two-attacker situation, supplying a cue about which piece to 
attend to provided an advantage in response time to less skilled players, but 
no advantage to expert players because the latter appeared to encode both 
attack relations simultaneously. 

In summary, experts rely on a rich network of chess patterns stored in 
long-term memory structures to give them a larger visual span when 
encoding chess positions. They encode chess information far more quickly 
and accurately than non-experts. Within the first second of exposure to a new 
position, experts are examining salient squares on the chessboard and 


extracting, in parallel, chess relationships critical to choosing good moves. 


Memory Processes 


As early as Binet (1966), and in particular in de Groot’s work, knowledge has 


been identified as a key component of chess expertise. In order to understand 
how knowledge (held in memory) mediates skill, a substantial amount of 
research has been carried out on chess players’ memory. Domains of interest 
include memory for static positions, memory for moves and sequences of 
moves (discussed in the section on blindfold chess), and the structure and 
contents of long-term memory (LTM), including the number of chunks 
necessary to reach expert performance. In many cases, experimentation has 


been carried out in concert with computational modeling. 


Memory Recall for Positions: Chase and Simon’s Key Results 


Extending de Groot’s study showing a striking skill effect in the recall of 
game positions, Chase and Simon compared behavior in a recall task and a 
copy task to identify the building blocks of chess knowledge. They proposed 
that skill did not reside in differences in short-term memory (STM) capacity 
or encoding speed, but in the number of chunks held in LTM (see Gobet & 
Simon, 1998, and Gobet & Clarkson, 2004, for replications). These chunks 
give access to information such as what move to play, what plan to follow, 
and what evaluation to give to (part of) the position. Thus, their theory 
explained both why masters choose better moves in spite of their selective 
search (because chunks enable them to identify the key features of a position, 
and guide search during look-ahead) and why they perform better in a 
memory task (because they can partition the position in relatively large 


groups of pieces, unlike weaker players who have to use smaller groups, 


which overtax STM). Some of these ideas were implemented in a computer 
program, MAPP (Simon & Gilmartin, 1973), which simulated recall up to 


expert level. 


Problems with the Chunking Theory Led to the Template Theory 


A number of experiments have helped refine Chase and Simon’s theory. 
Charness (1976) showed that the presence of an interfering task reduced 
recall only marginally, which runs counter to the assumption of a slow 
encoding in LTM. Others (Cooke et al., 1993; Frey & Adesman, 1976; Gobet 


& Simon, 1996a) have shown that players can remember multiple boards 


reasonably well, which highlighted a weakness of the original chunking 
theory. 

These results, as well as the fact that verbal protocols reveal that masters 
use larger structures than the chunks identified by Chase and Simon (e.g. de 
Groot, 1965; de Groot & Gobet, 1996; Freyhoff, Gruber, & Ziegler, 1992; 
Gobet, 1998a), led Gobet and Simon (1996a, 2000) to revise the chunking 


theory. Their template theory aimed to remedy these weaknesses, while 


keeping the strengths of the original chunking theory. It also aimed to show 
how high-level, schematic structures (templates) can evolve from perceptual 
chunks. As with the chunking theory, chunks and now templates are crucial 
in explaining how players access relevant information via pattern recognition. 

The computer program CHREST (Chunk Hierarchy and REtrieval 
STructures) implements aspects of the template theory. CHREST consists of 
an STM, an LTM indexed by a discrimination net, and a simulated eye. Each 
cognitive process has a time cost; for example, it takes 50 ms to place a 


chunk in STM, and 8 seconds to create a new chunk. During the learning 


phase, the program automatically acquires chunks and templates by scanning 
a database of positions taken from masters’ games. During the testing phase, 
it is placed in the same experimental situation as human participants. The 
program has simulated various characteristics of players’ eye movements (de 
Groot & Gobet, 1996), the details of reconstruction in recall experiments 
(Gobet & Simon, 2000; Gobet & Waters, 2003), as well as the way novices 
learn to memorize chess positions (Gobet & Jackson, 2002). Beyond chess, 
variants of the program have been applied to memory and problem-solving in 


Awale (an African “sowing” board game; Gobet, 2009) and Go (Bossomaier, 


Traish, Gobet, & Lane, 2012), memory for computer programs, use of 
diagrammatic representation in physics, concept formation, and children’s 


acquisition of language (Gobet et al., 2001). 


Random Positions 


While Chase and Simon (1973a, 1973b) found no skill difference in the recall 
of random positions, Gobet and Simon (1996b) did find such a difference, 
although the effect is weak and requires significant statistical power to detect. 
This skill effect remains across a wide range of presentation times (from 1 


second to 60 seconds; Gobet & Simon, 2000) and with positions where the 


location as well as the distribution of pieces is randomized (Gobet & Waters, 
2003). These results are consistent with the chunking and template theories 
that predict that strong players, who have more chunks, are more likely than 
weaker players to recognize some patterns even with random positions. These 
predictions, which were made by computer simulations with CHREST 
(Gobet & Simon, 1996b, 2000; see Figure 31.1), have been proven correct in 
other domains of expertise too (Sala & Gobet, 2017). 
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Figure 31.1 Top panels: a game position (left) and a random position 
(right). Bottom panel: percentage of pieces correctly replaced as a function 
of the type of position and skill level, both for humans and CHREST. 


Source: Fernand Gobet. 


New Estimates of the Vocabulary of the Master 


Based upon computer simulations with MAPP, Simon and Gilmartin (1973) 
estimated that one needed to acquire from 10,000 to 100,000 chunks to reach 
master level in chess. To test Holding’s (1985) prediction that a much smaller 
number was required if one assumed that the same chunk could encode the 


same constellation of pieces placed at different locations on the board, 


Saariluoma (1994) modified positions by swapping quadrants, and Gobet and 
Simon (1996c) modified positions by taking their mirror images along 
various axes of symmetry. Both found that these manipulations affected 
recall, which runs counter to Holding’s predictions but supports the original 
chunking theory. Computer simulations with CHREST show that at least 
300,000 chunks are required to reach grand master level recall performance, 
even with the presence of templates, which were not part of the chunking 


theory. 


Problem-Solving Processes 


De Groot (1965) asked his participants to think aloud when choosing their 
next move in a problem position. The quantitative and qualitative measures 
he extracted from the verbal protocols provided important empirical 
information about chess players’ thinking and have served until now as a 
source of ideas for experimental research and development of theories of 


heuristic search. 


Macrostructure of Search in Chess 


De Groot (1965) found few differences in the macrostructure of search 
between world-class grand masters and experts. Surprisingly, during their 
search, players from both skill levels tended to search at similar depth, to 
consider the same number of positions, and to propose similar numbers of 
candidate moves. However, the grand masters chose better moves than the 
experts, they generated moves faster, they reached a decision faster, and, 
during their search, they examined moves and sequences of moves that 
tended to be more relevant. 

Holding (1985) argued that de Groot’s (1965) small sample (five grand 
masters and five experts) may have concealed existing skill differences. 
Supporting Holding’s view, some skill differences were found with samples 
including weaker players (e.g. Charness, 19816; Connors, Burns, & 
Campitelli, 2011; Gobet, 1998b; Saariluoma, 1992). Charness (1981a) 
suggested that depth of search increases up to expert level, after which it 
stays uniform. Charness (1989) conducted a nine-year longitudinal 


investigation of a Canadian player who advanced (in power law fashion, 


though see Howard, 2014b, for issues on power fits) from an average level 
performance (1,600 rating points) to international master level performance 
(2,300 rating points) and found no significant increase in depth of search. 
However, international masters and grand masters sometimes carry out 


shallower search than masters (Saariluoma, 1990), perhaps indicating that 


they can tailor their search mechanisms to the demands of the position. Gobet 
(1997) carried out computer simulations with the SEARCH model (see 
below) and concluded that average depth of search keeps increasing with 
higher skill levels, but with diminishing returns (i.e. it follows a power law). 
Recently, researchers have identified two factors that affect depth of 
search, namely, the complexity of the problem and level of specialization. 
While strong players tend not to search much in familiar, strategic, and/or 
simple positions, they can search at great depths and breadths in positions 
that are unfamiliar, tactical, and/or complex (Bilalic, McLeod, & Gobet, 
2009a). In contrast, weaker players are not able to adapt their search strategy 
to the requirements of the task environment and must satisfy themselves with 
relatively shallow search in all types of positions. Being familiar with a type 
of position also facilitates search when necessary. Bilali¢ et al. (2009a) 
compared the problem-solving behavior of players who specialized in two 
different chess openings (openings are the first moves of a game, which 
players spend a great deal of time studying). They uncovered an interaction 
between players’ specialization and position types. When trying to find a 
move in positions belonging to their opening specialization, players tended to 
search deep and not as broadly. By contrast, when dealing with positions 
outside their specialization, players tended to show the opposite pattern: 


broad and relatively shallow search. Solving problems outside their opening 


specialization reduced player performance by 200 Elo points (one standard 


deviation in skill). The same decline occurred in a memory recall task. 


Selective Search, Move Generation, and Pattern Recognition 


De Groot found that all players were highly selective in their search, rarely 
visiting more than 100 nodes before choosing a move. Calderwood, Klein, 
and Crandall (1988) showed that masters can make relatively good decisions 
even under time pressure (about 5 s per move). Jastrzembski, Charness, and 
Vasyukova (2006) and Chassy and Gobet (2013) found that skilled players 
can rapidly and accurately identify a threat such as the presence of a check. 
Gobet and Simon (1996d) found that world champion Kasparov, when 
playing simultaneous games against teams consisting of up to eight 
international masters and grand masters, performed at a level that still placed 
him among the six best players in the world. They argued that, while 
Kasparov’s performance was weaker than in normal games and showed more 
variability, it was higher than theories mainly based on search would predict. 
Comparing the quality of play of world-class grand masters in standard 
games (about three minutes per move, on average) and rapid games (less than 
30 seconds per move, on average), Chabris and Hearst (2003) found that this 
decrease of thinking time by a factor of six increased the number of blunders 
per 1,000 moves by 40 percent (5.02 in classical games vs. 6.85 in rapid 
games). While they took this as evidence for the role of search, a more 
natural interpretation of the 6:1 change in search time leading to a 1.4:1 
increase in the number of blunders is that it better supports theories 


emphasizing pattern recognition. 


Proponents of search models often cite Holding and Reynolds’ (1982) 
experiment as evidence that search and pattern recognition can be 
dissociated. Holding and Reynolds, who used semi-random positions as 
stimuli, first asked players to recall the position after an eight-second 
presentation, and then to choose what they thought would be their move. 
They found that skill correlated with the quality of chosen move after a few 
minutes’ deliberation, but not with the recall or the evaluation after brief 
presentation. Schultetus and Charness (1999) extended Holding and 
Reynolds’ (1982) experiment with a crucial addition: they asked players to 
recall the position at the end of problem-solving. Like in the original study, 
stronger players did not recall the position better after five seconds, but chose 
better moves. However, they also obtained better results in the recall 
performance following problem-solving. Schultetus and Charness (1999) 
argue that these results are consistent with the hypothesis that pattern 
recognition underpins skill in chess. That is, in order to choose better moves, 
better players were able to form new relational patterns for the unusual piece 
placements. These new chunks provided the recall advantage after problem- 
solving. 

Consistent with the data we have presented in this section about pattern 
recognition and in the previous section about search, Moxley, Ericsson, 
Charness, and Krampe (2012) found that, although strong players found 
reasonably good moves rapidly, they found better moves on average when 
they had the opportunity to search. This result casts doubt on the application 
of the “take-the-first” heuristic as an explanation of expert decision making 
(Raab & Johnson, 2007). 


Progressive Deepening 


De Groot (1965) found that players were visiting the same branches of the 
search tree repeatedly, either directly or after visiting other branches. 
According to de Groot, this phenomenon of “progressive deepening” occurs 
both in order to compensate for limitations in memory and for propagating 
information from one branch of the search tree to another (de Groot, 1965; de 
Groot & Gobet, 1996). Gobet (1998b) found that skill affects how 


progressive deepening is carried out. The maximum number of immediate re- 


investigations of base moves was proportional to players’ strength, while the 
maximum number of non-immediate re-investigations was inversely 


proportional to players’ strength. 


High-Level Knowledge and Planning 


De Groot (1965), who emphasized the role of conceptual knowledge in chess 
expertise, reported that players’ descriptions of games were centered on key 
positions; this finding has been confirmed by other researchers (e.g. Cooke et 
al., 1993; de Groot & Gobet, 1996). The presence of these key positions 
enables masters to acquire what de Groot called a “system of playing 
methods,” many of which are stereotypical. By applying this routine 
knowledge, masters can often find good moves with minimal search. 
Saariluoma (1990, 1992) tested this hypothesis with tactical positions, and 
found that strong players tended to choose stereotyped solutions and missed 
shorter (but non-typical) solutions, thus displaying a typical Einstellung (set) 
effect. Bilali¢, McLeod, and Gobet (2008a, 2008b) as well as Sheridan and 
Reingold (2013) replicated and extended these results using eye-movement 


analyses, showing that, with advantageous Einstellung moves, players kept 


fixating the first (non-optimal) move that came to their mind, even though 
they were reporting that they were trying to find a better one (see Figure 
31.2). Saariluoma and Hohlfeld (1994) were interested in planning with 
strategic positions. They found that null moves (missing moves for one side) 
were common (about 12 percent of all moves). This result is similar to the 10 


percent found in a previous study by Chamess (1981a). While high-level 


knowledge is important, it should be noted that chess problem-solving is 
minimally affected by concurrent verbal activities unless they interfere with 
executive processes, suggesting that chess representations are more likely to 
be abstract and spatial than verbal (Robbins et al., 1996). 


Figure 31.2 Example Einstellung chess position (left side) and a “heat 
map” representation of eye fixation areas (right side) for an amateur and an 
expert player over the first and last quarter of a choose-a-move task. 
Amateurs were less able to extend their search beyond the top left quadrant 
of the board. 


We thank Sheridan and Reingold (2013) for providing the figure. 


Computational Models of Problem-Solving 


Simon and colleagues developed a number of process models of problem- 
solving in chess (see Newell & Simon, 1972). Gobet and Jansen (1994) 
described a program that uses pure pattern recognition to select moves, 
without carrying out any search. This was presented more as a first step 
toward a full problem-solving program than as a theory of human problem- 
solving. Gobet (1997) developed SEARCH, a probabilistic model that 
integrates the concepts of pattern recognition, search, and heuristics. This 
model, which was a direct implementation of the template theory and which 


incorporated insights from previous theories (e.g. de Groot, 1965; Newell & 


Simon, 1972), does not play chess but computes several measures such as 
depth of search, rate of search, and the level of fuzziness in the mind’s eye as 
a function of the skill level (i.e. number of chunks). Every cognitive 
operation has a time cost; for example, it takes two seconds to carry out a 
move in the mind’s eye, and ten seconds to evaluate a position using 
heuristics. 

The program predicted that depth of search follows a power law of skill. 
When simulating a small number of participants (as is typical in chess 
research), the program also showed substantial variability, as was found in 
Saariluoma’s (1992) study, where international masters and grand masters 


searched less than weaker masters. 


Blindfold Chess 


A number of researchers have investigated blindfold chess experimentally. In 
blindfold chess, a player carries out one or several games without the view of 
the board and the pieces; the moves are communicated through standard 
chess notation. 

Saariluoma (1991) studied memory for move sequences using blindfold 
chess. He dictated one move every two seconds, from three types of 
sequences: moves actually played in a game, random but legal moves, and 
random and possibly illegal moves. He found that masters could recall almost 
perfectly the position for the moves taken from actual games and legal 
random moves after 15 moves, but performed poorly with illegal random 
moves. With additional moves, the recall of legal random moves decreased 
much faster than that of game moves. 

Saariluoma (1991) and Saariluoma and Kalakoski (1997) presented one 
or several games aurally (dictating moves using the standard algebraic chess 
notation) or visually (presenting only the current move on a computer 
screen). Only a few of their results can be presented here: blindfold chess 
requires mainly visuospatial working memory, and makes little use of verbal 
working memory; differences in LTM knowledge (e.g. number of chunks) 


rather than differences in imagery ability underpin skill differences; abstract 


representations are essential (see also Binet, 1966); and there is no difference 
between auditory and a visual presentation. Campitelli and Gobet (2005) used 
blindfold chess to study how perception filters out relevant from irrelevant 
information. They found that irrelevant information affects chess masters 


only when it changes during the presentation of the target game. 


Problem-solving has also been studied using blindfold chess 


(Saariluoma & Kalakoski, 1998). In a task consisting in searching for the best 


move, they found that players memorized pieces better when these were 
functionally relevant. This difference disappeared in a task where players had 
to count the number of pieces. They also found that tactical combinations 
embedded in a game position were easier to solve than those contained in a 
random position. As with normal chess, visuospatial interfering tasks 
negatively affect problem-solving performance. Finally, Chabris and Hearst 
(2003) found that the number of blunders did not increase much when grand 
masters played blindfold games as compared to games with a view of the 
pieces. Campitelli and Gobet (2005) argue that most of the results found on 
blindfold chess can be explained by the template theory. 


Risk 
Selecting a move implies a certain amount of risk, and several authors have 
focused on this aspect of chess playing, all analyzing large databases of 
games and inferring risky and non-risky strategies from the first moves 
played in games. Given this narrow definition, these studies have a different 
flavor than most of those carried out on chess problem-solving. Gerdes and 
Grdnsmark (2010) found that females tended to be more risk-averse than 
males. Slezak and Sigman (2012) observed that players tend to use less risky 
strategies when playing against stronger players in rapid-transit games, which 
turns out to be a suboptimal approach. Finally, Chassy and Gobet (2015) 
estimated risk-taking between groups of countries, as inferred by risk-taking 


from chess players representing them. 


Building a Human Master 


Prodigies: Born or Made, and the Issue of Critical Periods 


This section contains a brief review of topics related to potential 
contributions of talent and practice to skill. A fair conclusion from the 
available evidence is that we still do not have data rich enough to determine 


how they might interact in the development of chess expertise. 


Developmental Issues 


In a classic study on the role of knowledge on memory development, Chi 
(1978) found that, while non-chess-playing adults were better at memorizing 
digits than chess-playing children, they were worse at memorizing game 
positions. Thus, domain-specific knowledge can override developmental 
differences. Schneider, Gruber, Gold, and Opwis (1993) extended Chi’s study 
by adding child novices and adult experts to the design; they also presented 
random positions and added a non-chess visuospatial control task. Adults and 
children offered the same pattern of results: experts’ superiority was the 
largest with meaningful positions, was reduced with the random positions, 
and all but disappeared with the board control task (while absent in the first 


trial, skill effects were apparent in later trials). 


Learning 


Several longitudinal studies have trained novices to memorize chess positions 
(Gobet & Jackson, 2002; Saariluoma & Laine, 2001). Typically, learning 
follows a power function. Computer models based on chunking could 
simulate the data relatively well (Gobet & Jackson, 2002; Saariluoma & 
Laine, 2001). 


Didierjean, Cauzinille-Marméche, and Savina (1999) were interested in 
how chess novices use reasoning by analogy in learning to solve chess 
combinations (smothered mates). The results show that transfer was limited 
to problems perceptually similar to the examples and did not extend to 
problems requiring the use of the abstract principle behind the solution of 


these problems. 


Training, education, and transfer 


Given the importance of deliberate practice in an entrepreneurial domain such 
as chess, one could expect that powerful training methods have been 
developed. There is not much about this topic in the literature, however. 
Gobet and Jansen (2006) show how educational principles that can be used in 
chess training can be derived from the template theory. The necessity of 
having a coach is debated in the literature; for example, Charness, Krampe, 
and Mayr (1996) found a bivariate but not a unique multivariate correlation 
between chess skill and the presence of a coach in one sample; however, 
Charness, Tuffiash, Krampe, Reingold, and Vasyukova (2005) did find it in 
another. Gobet, Campitelli, and Waters (2002) note that using computer 
databases and playing computers may provide more efficient training tools 
than traditional training practice based on books, which is consistent with the 
progressive replacement of the latter by the former in professional practice. 
This change in training practice techniques may well explain Howard’s 
(1999) observation that the number of young players among the world’s elite 
has increased in the last decades, which he takes as evidence that average 


human intelligence is rising overall. Another explanation is that as young 


player populations increase, the best-trained individuals should reach higher 
levels of performance. 

Do skills acquired in playing chess transfer to other domains? The 
answer seems to be: not much. A first source of evidence is experimental; it 
shows that transfer is very difficult. For example, chess players’ perceptual 
skills do not help them to memorize visual shapes unrelated to chess (Waters, 
Gobet, & Leyden, 2002) and chess planning skills do not transfer to solving 
the Tower of London task (Unterrainer, Kaller, Leonhart, & Rahm, 2011). A 
second source of evidence comes from the effects of chess instruction on 
academic domains and general cognitive abilities. A recent meta-analysis 
(Sala & Gobet, 2016) found a moderate overall effect size (g = 0.34) of chess 
instruction. There was also an effect of duration of treatment, studies with at 
least 25 hours of instruction obtaining stronger effects than shorter studies. 
However, these conclusions are limited by the fact that studies controlling for 


placebo effects were nearly absent. 


Individual Differences 


Burgoyne et al. (2017) carried out a meta-analysis on the relationship 
between chess skill and cognitive ability. Based on 19 studies, they found 
that chess skill had positive correlations with fluid reasoning (r = 0.24), 
comprehension-knowledge (r = 0.22), short-term memory (r = 0.25), and 
processing speed (r = 0.24). Age and skill were significant moderators for the 
correlation between chess skill and fluid reasoning: the correlations were 
stronger for youth samples compared to adult samples, and for unranked 
samples compared to ranked samples. Note that all the studies reviewed in 


this meta-analysis used quasi-experimental designs; therefore, the results are 


based on correlations, which are equivocal about the direction of causality (is 
intelligence a prerequisite to chess skill, or does chess playing improve one’s 
intelligence?). The differing patterns between children and adults are 
consistent with developmental theories that propose differentiation of abilities 
across time. Early in development all forms of problem-solving are 
dependent on fluid intelligence (search) but later, crystallized intelligence 
(knowledge: templates and chunks) changes the way that problem-solving is 
carried out. 

Several researchers have examined chess players’ personality. Bilalic, 
McLeod, and Gobet (2007) found that children with high scores on 
intellect/openness and energy/extraversion were more likely to play chess, 
while the opposite was true with children scoring higher on agreeableness. 
Vollstadt-Klein, Grimm, Kirsch, and Bilali¢ (2010) found that adult male 
players were not different from the population norms, but female players had 
higher life satisfaction, fewer physical complaints, and higher achievement 
motivation than the norms. Introversion was positively correlated with skill in 
male players, but negatively with female players. Compared to a predictor 
such as deliberate practice, there are weak relationships between personality 


and expert performance (Charness, Tuffiash, & Jastrzembski, 2004). 


Gender Differences 


Men vastly outperform women in chess; for example, only about 1 percent of 
grand masters are women, and there is only one female (in 2018) among the 
best 100 chess players (Chinese grand master Hou Yifan). Several authors 
have investigated the possibility that these differences are due to participation 


rates. Charness and Gerchak (1996) developed a statistical method to predict 


the highest score of a population and showed that it estimated reasonably 
well the difference between the top male and female player of the time (Gary 
Kasparov and Judith Polgar). Extending this method, Bilali¢, Smallbone, 
McLeod, and Gobet (2009b) showed that participation rates could account for 
96 percent of the differences between the 100 male and female chess players 
in the German rating list. Finally, Chabris and Glickman (2006) analyzed the 
ratings of the US Chess Federation over 13 years. They found that more boys 
started competition than girls, and that their initial performance was higher. 
When boys and girls of equal strength are considered, they progressed and 
dropped out at the same rate. Similarly, in geographical locations where girls 
represented at least half of the players, there are no differences between boys’ 
and girls’ initial performance. Chabris and Glickman took these results as 
support for the role of participation rates. 

Howard (2014a) adduced data showing that the participation rate 
hypothesis does not provide the full explanation for gender differences in 
chess skill, and that there might be other factors, such as innate biological 
differences. He compared gender differences in performance in countries that 
had different percentages of female participation. In spite a large variations of 
female participation, ranging from 4.2 percent to 32.3 percent, the difference 
of performance between males and females remained nearly constant. Other 
explanations have been investigated as well. Maass, D’Ettole, and Cadinu 
(2008) explored the role of gender stereotypes. When playing games through 
the Internet against men, women played less well when a gender stereotype 
was primed. Women also displayed a lower chess-specific self-esteem. De 
Bruin, Smits, Rikers, and Schmidt (2008) found that deliberate practice was a 


much better predictor of performance than gender. 


The Role of Deliberate Practice and Tournament Experience 


As appears to be true in other domains, skill acquisition in chess requires a 
considerable investment. Few players reach master level performance with 
less than 1,000 hours of serious study (Charness, Krampe, & Mayr, 1996). 
Relying on responses to retrospective questionnaires, these investigators 
probed a large sample of tournament players from different countries 
focusing on how much time they spent in serious study alone (deliberate 
practice) versus that spent in tournament play and analysis of games with 
others. Other predictors for current skill level included variables such as 
current age, starting age, age when serious about chess, age when joining a 
chess club, presence of coaching, and size of chess library. The variables 
making independent contributions to explaining current chess rating were 
serious study alone, size of chess library, and current age. Tournament play 
was not statistically significant after taking deliberate practice time into 
account. Not surprisingly, age was a negative predictor (older players tended 
to have lower ratings, averaging a loss of about 5-6 rating points per year), 
whereas deliberate practice and size of chess library were strong positive 
predictors, accounting in combination for nearly 70 percent of explained 
variance in current rating. 

In an enlarged version of the first sample and in a new sample, Charness 
et al. (2005) showed a somewhat different pattern of relationships, with both 
coaching and tournament play in addition to deliberate practice making 
independent predictions to current chess rating. For predicting a player’s peak 
rating, the two practice variables accounted for most of the variance. Of 


course, this correlates approach suffers from the weakness that causality is 


not identifiable. Longitudinal research is needed to trace how process 
variables co-vary with changes in rating. 

Later studies confirmed the role of deliberate practice but also indicated 
the importance of other factors. Gobet and Campitelli (2007) found that skill 
level correlated moderately with the cumulative amount of individual practice 
(r = 0.42). In addition, there was considerable inter-individual variability. On 
average, players needed 11,053 hours of deliberate practice to become 
masters; however, one player needed only 3,016 hours, another needed 
23,608 hours (a 1:8 ratio), and some players devoted more than 25,000 hours 
without becoming masters. (See also Hambrick et al., 2014, on the issue of 
variability.) Campitelli and Gobet (2008) found two results that did not 
support Ericsson et al.’s (1993) assumption that performance is a monotonic 
function of practice. First, after three years of serious chess play, masters and 
candidate masters had accrued the same number of deliberate practice hours, 
but masters’ rating was higher. Second, candidate masters did not see much 
improvement after three years, despite continuing deliberate practice. 

A central tenet of deliberate practice theory is that for individuals to 
advance in skill they need to find practice activities to circumvent current 
limitations. Presumably, candidate masters were failing to engage in the right 
types of practice to advance. However, deliberate practice theorists need to 
characterize component skills in chess much more precisely in order to 
specify what constitutes effective practice for an individual. Questionnaire 
studies asking for hours of practice are blunt instruments in this regard. 

Starting age plays an important role, beyond simply allowing individuals 
to amass a higher amount of deliberate practice. In support of Elo’s (1986) 
hypothesis that there is a critical period in the acquisition of chess skill, 


Gobet and Campitelli (2007) uncovered a negative correlation between 


starting age and chess skill (r = —0.37), a correlation that is still reliable after 
the amount of deliberate practice is controlled for with partial correlation (r = 
—0.40). Twelve years of age appears to be a tipping point: players who began 
to play chess earnestly at or before 12 years were 24 per cent likely to reach 
an international level; by contrast, players who began after 12 were only 2 
percent likely to reach this level. 

De Bruin, Kok, Leppink, and Camp (2014) studied a group of 24 young 
novices attending a chess course over ten weeks. They found that chess 
performance was affected separately by IQ and practice, and that motivation 
had an indirect effect by moderating practice. 

De Bruin et al. (2008) studied the relationship between deliberate 
practice and the likelihood of dropping out, using a sample of young elite 
Dutch chess players. As in previous studies, deliberate practice was a strong 
predictor of chess performance during players’ entire career, which the 
authors took as support for the monotonic benefits assumption of deliberate 
practice. Dropouts tended to improve more slowly over time, mostly due to 
the fact that they practiced less. The amount of deliberate practice was 
predicted by competitiveness and the will to excel, and dropouts tended to 
have lower achievement motivation and chess-specific motivation than 
individuals who kept playing (de Bruin, Rikers, & Schmidt, 2007). 

Individual differences raise the question of talent. Two studies compared 
current world champion Magnus Carlsen to less skilled players. Howard 
(2011), estimating the amount of practice through biographies, showed that 
Carlsen, in spite of having much less practice than the Polgar sisters, whose 
career is often taken as support for the role of practice, progressed much 
quicker and obtained much higher top ratings. Gobet and Ereku (2014) 
analyzed the June 2014 rating list published by the World Chess Federation, 


finding a remarkable 66-point gap between then 23-year-old Magnus Carlsen 
and the next player. The next ten players clustered closer together. Using 
biographical estimates of practice, Carlsen had significantly less estimated 
deliberate practice (18 years) than the following ten grand masters (24.6 
years) despite starting serious play later (age 8). These results are inconsistent 
with the hypothesized monotonic relationship between deliberate practice and 
performance. Potential mechanisms for explaining talent could be faster 
chunking rates (e.g. in CHREST, 5 s versus 8 s to consolidate a new chunk) 
or representation differences, such as development of better evaluation 
functions associated with templates. 

Two potential biological markers of chess talent have been suggested: 
handedness and seasonality of birth. Cranberg and Albert (1988) and Gobet 
and Campitelli (2007) found that about 18 percent of male chess players were 
not right-handers. This percentage is reliably higher than that in the general 
population (from 10 percent to 13.5 percent). However, in both studies, skill 
level did not correlate with handedness. Gobet and Chassy (2008) found that 
chess experts in Northern-hemisphere countries are more likely to be born in 
the first half of the year (52.3 percent) than in the second half (47.7 percent). 
They speculated that this pattern is due to external factors more likely to be 
present in late winter and early spring, such as viruses, which affect the 
normal developmental trajectory of the prefrontal cortex and thus the 
likelihood to excel in chess. 

It seems likely that (deliberate) practice and other individual differences 
(e.g. motivation, cognitive ability) play roles in the acquisition of chess 
expertise, in addition to factors such as characteristics of the environment, 
cultural and historical contexts, and even luck (Charness, Krampe, & Mayr, 


1996; Gobet, 2013, 2016). These factors may interact in complex ways (e.g. 


Simonton, 1997) and as a function of time (e.g. availability of chess literature 


and strong competition). Gobet (2013) argues that, given the multiple 
variables and interactions involved, the use of formal modeling is necessary 
to understand the acquisition of (chess) expertise. Campitelli, Gobet, and 
Bilali¢ (2014) addressed this issue by developing the practice-plasticity- 
processes model, which is expressed as mathematical equations relating 
cognitive processes (domain-specific pattern recognition and heuristics), 
practice, and neural plasticity. Simulations were able to successfully capture 
phenomena such as the existence of young players with top-level 
performance, the presence of a critical period for learning chess, and the 


decline of chess skill in older adults. 


Conclusions 


We have reviewed a combination of empirical and theoretical work that has 
identified and characterized the processes involved in choosing a good move 
in chess. Many phenomena identified are central to the concerns of 
psychology, particularly to theories about individual differences, memory 
systems, developmental processes (Krampe & Charness, Chapter 42, this 
volume), and theories in cognitive science and neuroscience (Bilalic & 
Campitelli, Chapter 14, this volume). Simulation work has proven useful in 
describing how aging processes interact with knowledge processes to predict 
memory performance (Mireles & Charness, 2002; Smith, Gobet, & Lane, 
2007). 

There are several reasons for the unique impact of chess on expertise 
research. First, chess has an ecologically valid and quantitative measure of 
skill. Second, as a closed environment, it makes it easier to design 
experiments that specifically address theoretical questions. In general, the 
answers that have been found tend to generalize to other domains of 
expertise. Examples include the role of progressive deepening, the 
importance of pattern recognition and search, and experts’ superior memory 
for meaningful and even random domain-specific material. Finally, more so 
than in any other domain of expertise, detailed computational models have 
been developed to account for empirical results. Contrasting with nearly all 
other domains of expertise where informal and often vague theories are 
developed, chess has enjoyed precise theories that make clear-cut empirical 


predictions. As these theories use general mechanisms, for example chunking 


and heuristic search, they have shed light on cognition beyond expertise per 
se. 

Nonetheless, many issues remain unresolved. It is not yet clear how 
deliberate practice and cognitive abilities jointly determine performance 
across the lifespan given the differing patterns seen in children, young adults, 
and older adults. Tighter links still need to be drawn between perceptual 
processes and search processes, including how evaluation functions develop 
with skill. With the ready availability of modern tools (neuroimaging, eye 
tracking, database analysis) in conjunction with reliable older ones (think- 
aloud protocol analysis and computer simulation), the future seems bright for 


expanding our knowledge of expertise in chess. 
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Introduction 


Competence in mathematics is a basic requirement for effective citizenship in 
a modern numerate society (Cockcroft, 1982). Poor numeracy skills are 
known to be a serious handicap for paid employment in the US (Rivera-Batiz, 
1992) and the UK (Bynner & Parsons, 1997). The UK Basic Skills Agency 
has published a report suggesting that numeracy is more important than 
literacy in terms of career prospects in the UK (Bynner & Parsons, 1997). 
The trend is toward an even greater emphasis on numeracy; recent research 
for the British Science, Technology and Mathematics Council shows that 
“mathematical skills in the workplace are changing, with increasing numbers 
of people engaged in mathematics-related work, and with such work 
involving increasingly sophisticated mathematical activities” (Hoyles, Wolf, 
Molyneux-Hodgson, & Kent, 2002, p. 6). 

The level of competence routinely demanded in numerate cultures today 
would have been considered quite exceptional 200 years ago. How then does 
one distinguish today’s expert from the normally competent school-leaver 
who can handle numbers of arbitrary size, fractions and decimals, logarithms, 
equations with unknowns and negative roots, and some differentiation and 
integration? One could arbitrarily take the top n percent of a standard test 
(like the SAT-M: Scholastic Aptitude Test — Mathematics), but what should n 
be? Francis Galton, in Hereditary genius, used obituaries from The Times of 
London and a biographical dictionary, Men of our time, as the criteria of 
“eminence.” This gave him an estimated proportion of 0.025 percent of the 


population. Really exceptional individuals, his class G, were about one- 


twentieth of these. He even designated a class X of people who were fewer 
than one in a million (Galton, 1979). 

However, not every expert that we would wish to consider has taken the 
SAT-M, or been the subject of a Times obituary. The important criterion is 
that the candidate expert demonstrates “reproducible superior performance,” 
preferably under something like laboratory conditions (Ericsson & Charness, 
1994). 

There have been three extensive early reviews of exceptional 
mathematical abilities. The first was by E. W. Scripture, a psychologist at 


Leipzig (Scripture, 1891). He reviewed in some detail the lives of 12 


“arithmetical prodigies,” including one mathematician of note, Carl Friedrich 
Gauss. From a “psychological analysis” of these lives, he identified five 
characteristics that seemed to distinguish the prodigies: the accuracy and 
“rapidity” of memory; “arithmetical association” (knowing lots of 
arithmetical facts and procedures); inclination; mathematical precocity; and 
“imagination” (visual imagery). 

The second review, like Scripture’s, was published in the American 
Journal of Psychology, and was written by Frank D. Mitchell of “The 
Psychological Seminary of Cornell University” (Mitchell, 1907). Also like 
Scripture, he reviews the lives of prodigies, the same ones as Scripture plus a 
further case, the author. These are summarized in a table that lists the 
heredity, development, education, mental calculation, and memory of each 
prodigy. Mitchell takes the view that prodigious abilities grow out of 
counting, a verbal skill where the numbers are recited out loud initially, and 
internally thereafter. This has an implication for the “memory type” used by 
prodigies. In his own case and that of most of the prodigies he analyzed, the 


memory is of the auditory type, reflecting this early experience since most of 


us begin to learn numbers through counting, which involves spoken words. 
Only between two and four of the prodigies he examined had memory of the 
“visual type.” 

The third review, by Alfred Binet, originator of the intelligence test 


(Binet, 1894), included reaction-time tests of two theatrical calculators of his 


day. Barlow (1952) has written about mathematical prodigies in the context 
of other kinds of prodigy. And there is an excellent reconsideration of the 
calculators studied by Scripture, Binet, and Mitchell, along with useful data 
on more recent calculators by Steven B. Smith in The great mental 
calculators: The psychology, methods, and lives of calculating prodigies 
(Smith, 1983). I shall be drawing heavily on this volume along with 
Scripture, Mitchell, and Binet for biographical details of calculating 
prodigies. Finally, one of the leading modern investigators has published a 
review of expert calculators (Pesenti, 2005). 

Calculators have attracted the attention of experimental psychologists 
usually focusing on a single case. The Polish calculator Salo Finkelstein was 
studied by Weinland and Schlauch (1937), who used sophisticated analyses 
of careful timing data from various mathematical tasks, but with no control 
participants. The British mathematician Alexander Aitken was studied by 
Hunter (1962), and there is a recent case study of Riidiger Gamm by Pesenti 
and colleagues (Pesenti et al., 2001). Only Binet (1894) seems to have 
explored the general phenomenon, carrying out experiments on two 
professional theatrical calculators and comparing their results with other 
groups of practiced and unpracticed calculators. He took considerable pains 
to find the optimal way of timing the stimulus presentation and the response, 


in an age when there were no voice-activated relays. However, he was less 


careful in designing the experiments so that all the participants received the 
same stimuli under the same conditions. 

The research on mathematical, especially numerical, abilities in general 
strongly suggests the existence of a domain-specific capacity. There are two 
main arguments in support of this. First, in the human brain there is a 
specialized network dedicated to numerical processing that both supports 
simple calculation and is part of a wider network for expert calculation (see 
below). Second, there is evidence for an innate basis to this specialization. 
One strand of evidence supports the claim for numerical capacity in infants, 
even in the first few months of life, when neither language nor frontal lobe 
functions such as reasoning have developed. They are able to respond 
discriminatively on the basis of numerosity (Antell & Keating, 1983; Starkey 
& Cooper, 1980) and can even mentally manipulate numerosities at six 
months by working out what would happen when an object is added or 
subtracted from an array (Wynn, 1992, 2000, 2002; Wynn, Bloom, & Chiang, 
2002). These infant capacities are similar to ones observed in monkeys 
(Hauser, MacNeilage, & Ware, 1996), and the neural basis of monkey 
numerical abilities is homologous with the regions active in monkey brains 
when doing numerical tasks (Nieder & Dehaene, 2009). This finding suggests 
ancestral versions that may have evolved because the ability to recognize the 
numerosity parameter in the environment offers advantages in foraging, 
mating (Edwards, Alder, & Rose, 2002), and also in conflict with 
conspecifics (McComb, Packer, & Pusey, 1994). (See Butterworth, 1999, for 
a review.) Moreover, developmental disorder can lead to selective deficits in 
the acquisition of even these simple numerical concepts when intelligence, 
memory, and language are all at normal levels (Landerl, Bevan, & 
Butterworth, 2004). 


However, even if it is accepted that we humans have inherited a 
specialized capacity for representing numerosities, it does not follow that this 
is normally distributed, such that some people have it to a greater degree than 
others, like height or IQ. It may be more like color vision — either it is 
normal, or it is defective in one of a small number of ways. In the same way 
that ability as a colorist may require normal color vision, the range of colorist 
abilities is not determined by better or worse color discrimination, so it is 
possible that calculating abilities require a normal numerical “starter kit,” but 
the abstract and complex skills that make expert mathematicians and 
calculators are built by other means on this basis. Smith (1983) compares it to 
juggling: “Any sufficiently diligent nonhandicapped person can learn to 
juggle, but the skill is actually acquired only by a handful of highly motivated 
individuals” (p. 6). Calculating prodigies themselves often say that their 
abilities come from their interest in numbers rather than from some special 
gift. 

So, my central question in this chapter is whether mathematical 
expertise, and in particular calculation expertise, depends on high cognitive 
abilities in non-numerical domains, such as reasoning and memory, or 


whether it can exist as a domain-specific achievement. 


What Makes for Mathematical Expertise? 


Francis Galton was quite clear that any kind of eminence depended on 
“natural ability,” which was by and large inherited. “By natural ability, I 
mean those qualities of intellect and disposition, which urge and qualify a 
man to perform acts that lead to reputation. I do not mean capacity without 
zeal, nor zeal without capacity, nor even a combination of both of them, 
without an adequate power of doing a great deal of very laborious work” 
(Galton, 1979, p. 37). 

Having compared divines, wrestlers, men of science, painters, poets, and 
others, he found that eminent people tended to have eminent parents and to 
produce eminent offspring. Of course, we are talking mostly about eminent 
men and their fathers, since it was difficult for women to achieve eminence. 
Overall, 31 percent of eminent men had an eminent parent. Some 26 percent 
of scientific men had scientific fathers, but 60 percent of scientifically 
eminent fathers have eminent sons (vs. 41 percent average). Galton 
speculates “descendants [are] taught ... not to waste [their] powers on 
profitless speculation” (p. 320). 

Although inherited characteristics are held to be the key, “it may be well 
to add a few supplementary remarks on the small effects of a good education 
on a mind of the highest order. A youth of abilities G, and X, is almost 
independent of ordinary school education” (p. 43). He gives as an example 
D’Alembert, who was a “foundling ... put out to nurse as a pauper baby to 
the wife of a poor glazier. The child’s indomitable tendency to the higher 


studies, could not be repressed by his foster mother’s ridicule and dissuasion, 


nor by the taunts of his schoolfellows, nor by the discouragements of his 
schoolmaster, who was incapable of appreciating him, nor even by the 
reiterated deep disappointment of finding that his ideas, which he knew to be 
original, were not novel, but long previously discovered by others” (pp. 
43-44). He records many other eminent men with a comparably unpromising 
history. 

He pays particular attention to “Wranglers,” students of mathematics at 
Cambridge University. Not only do these men have to pass the entrance 
examinations to Cambridge, but they were ranked from the highest (Senior) 
to the lowest strictly according to the marks obtained in an exam. These 
ranged from less than 500 to over 7,500. He was thus able to compare the 
Senior with the Second Wrangler, and he noted the enormous discrepancy 
often observed, with the Senior frequently achieving double the marks of the 


Second. 


Intelligence 


Ability in mathematics is widely seen as a marker for intelligence, and 
disability in mathematics in school is seen as a marker for low intelligence. 
This is not the place to discuss the general relationship between intelligence 
and mathematical ability but only to point out that severe disability can co- 
occur with good to superior IQ scores (Landerl et al., 2004). 

Shakuntala Devi (born 1940) is in the Guinness Book of World Records 
for being able to multiply two 13 digit numbers in 28 seconds. She was tested 
formally by the psychologist A. R. Jensen, who showed that she was not 
much better than average on standard IQ tests, and was actually slower than 
average on some tests of speed of mental processing, which Jensen regards as 
a reliable measure of intelligence (Jensen, 1990). Dehaene comments that 
therefore “Devi’s calculations were obviously not due to a global speed up of 
her internal clock: Only her arithmetic processor ran with lightning speed” 
(Dehaene, 1997, p. 165). 

Hermelin and O’Connor (1990) reported a young man who was able to 
recognize and generate primes of up to five digits four or five times faster 
than a graduate with a math degree and factorize these numbers faster and 
more accurately. What is extraordinary is that the man had a measured IQ of 


67 and was unable to speak or understand speech. 


Perception 


Here I consider whether expertise leads to experts “seeing” numerical stimuli 
differently from non-experts. The evidence is rather slim on this, but some 
recent research suggests that they do. This is perhaps not surprising since we 
know that chess grand masters see the board differently from moderate 
players (de Groot, 1965). 

In a very simple perceptual task where one has to select the larger of two 
arrays of dots, one’s ability to do this correlates with arithmetical expertise: 
that is, the better you are at selecting the larger array, the better your 


arithmetic (Halberda, Ly, Wilmer, Naiman, & Germine, 2012; Halberda, 


Mazzocco, & Feigenson, 2008). It is claimed that the task of comparing dot 
arrays taps into a process for extracting numerosity from the environment, 
which is regarded as a precursor and foundation for later arithmetical learning 
(Feigenson, Dehaene, & Spelke, 2004). However, the causal relationship 
between the two skills is controversial. It may be that good visual perceptual 
abilities are conducive to good arithmetic, and that improving them will 
improve arithmetic (Park & Brannon, 2013); or it may be that practice with 
arithmetic somehow improves these perceptual abilities (Piazza, Pica, Izard, 
Spelke, & Dehaene, 2013). There is also the problem that other perceptual 
judgments, such as matching the orientation of gabor patches, a completely 
non-numerical task, also correlate with arithmetical ability (Tibber et al., 
2013), suggesting, but certainly not demonstrating, that good visual 
perception may be an aid to developing good number skills. 


It has been proposed that: 


With appropriate practice, this information extraction will become faster 
and more automatic. The automatization of basic information pickup 
paves the way for the discovery of even more complex relations and 
finer detail, which in turn becomes progressively easier to process ... 
This cyclic process can be a positive feedback loop: Improvements in 
information extraction lead to even more improvements in information 
extraction. The resulting abilities to see at a glance what is relevant, to 
discern complex patterns and finer details, and to do so with minimal 
cognitive load, are hallmarks of expertise in all domains where humans 


attain remarkable levels of performance. 


(Kellman & Massey, 2013, p. 120) 


Kellman and colleagues have shown how perceptual learning can have a 
major impact on higher level, and more abstract, mathematical skills. 


Kellman trained a group of 8th and 9th graders in Algebra I courses. 


Students at this point in their learning show a characteristic pattern. 
Given simple equations to solve, such x + 4 = 12, accuracy is high, with 
an average across participants of around 80% correct solutions. 
Remarkably, however, students at this stage take an average of about 28 
s per problem! This pattern suggests that conventional instruction does a 
good job of addressing the declarative and procedural aspects of solving 
algebraic equations. Students know they should “get x alone on one 
side,” and “do the same operation to both sides of the equation,” and 
they were able to accomplish these goals with high accuracy. Their 
response times, however, suggest that we may underestimate the seeing 
problem in algebra learning. Someone with much greater experience 


looks at x + 4 = 12 and sees the answer at a glance. This kind of ability 


can reach higher and higher levels, supporting greater expertise, as 


illustrated in this example: 


_ (49 — 2¢y) 
(2 — w)(2¢) 


Given that this is a single equation with two unknowns, one might think 
at first glance that the problem does not permit a numerical solution for 
pt, but a more practiced observer may easily see that the equation permits 
easy simplification, and p = 1. In this case, even the relative 
unfamiliarity of the symbols used may make the seeing problem harder. 


Without changing anything mathematical, compare 


If this equation still has you reaching for pen and paper, seeing the 


structure may be better illustrated in this simpler version: 


(x — xy) 


(x)(1 — y) 


m= 


These examples all involve the distributive property of multiplication 
over addition. However, being able to enunciate this property would not 
produce fluent recognition of the distributive structure. Conceptually, 
and even computationally, these examples are all very similar, but you 
may have noticed the relevant structure more easily in one case than 
another. Improved encoding of relevant structure and potential 
transformations in equations is a likely result of PL [perceptual 


learning], one that is difficult to address in conventional instruction. 


(Kellman & Massey, 2013, pp. 125-126) 


Perceptual training “produced dramatic gains for virtually all students on 
this task, with accuracy changing from about 57% on initial learning trials to 
about 85% at the end of PLM usage, and response times per problem reduced 
by about 55%, from nearly 12 s per problem to about 7 s, suggesting the 
development of fluency in processing symbolic structure of equations” 
(Kellman & Massey, 2013, p. 127). 


Memory 


“The distinction often made between memory and calculation, with the 
implication that the great calculator is simply a little calculator with a big 
memory, using the same methods as his lesser rivals, is misleading; the 
process is always (in the ‘natural calculators’) a true calculation, and memory 
for figures is important only in so far as it stands in the service of calculation” 
(Mitchell, 1907, p. 132). 

Scripture (1891) distinguished “accuracy” and “rapidity” of memory 
from what he called “association.” We would now call the former “working 
memory” and the latter long-term “semantic memory” (Cappelletti, 
Kopelman, & Butterworth, 2002). Calculators themselves stressed the 
importance of both being able to hold many items in mind as they were 
carrying out calculations, and also knowing many more facts about numbers 


than the average person. 


Working Memory 
According to Smith (1983), George Parker Bidder (1806—78), an exceptional 


calculator and a leading engineer of his time (a collaborator with locomotive 


engineer, Robert Stephenson), was the first to explicitly draw attention to 
working-memory limitations on calculation. Bidder noted that, “As compared 
with the operation on paper, in multiplying 3 figures by 3 figures, you have 
three lines of 4 figures each, or 12 figures in the process to be added up; in 
multiplying 6 figures into 6 figures, you have six lines of 7 figures, or 42 
figures to be added up.” In general, the difficulty in using the mental 
analogue of the written method increases by something in the order of n? + n 


of the number of digits in an n * n problem (Smith, 1983, p. 53). More 


generally, the number of mathematical operations required to add two 
numbers grows linearly with the number of digits involved, but in 
multiplication using simple arithmetic the number of operations grows with 
the square of the number of digits. 

For this reason, it is very important for the calculator to develop 
techniques for reducing current load. Given that a three-digit number was, for 
Bidder (and most calculating experts), a single item, a three-digit by three- 
digit calculation, working from the left (instead of the right, as is normal in 
the written method), requires no more than five items to be currently 
maintained. For the problem 358 « 464, assuming trailing zeros are stored at 
no cost, Bidder probably worked it out as shown in Table 32.1 Although 
there are many steps, the current load is kept small, and the routine is easy to 


practice. 


Table 32.1 Reconstruction of the stages in George Bidder’s memory 


updating when calculating the solution to 358 x 464. 


(adapted from Smith, 1983, p. 54) 


Number of items 


Step 
1 
2 
3 


4 


Numbers in 


memory 


358 464 


120 000 


20 000 


140 000 


3200 


143 200 


18 000 


161 200 


3000 


164 200 


480 


164 680 


1200 


165 880 


to be 
maintained 


2 


3 


4 


3 


Calculation 


400 * 300 


400 * 50 


120 000 + 20 000 


400 ° 8 


140 000 + 3200 


60 * 300 


143 200 + 18000 


60 * 50 


161 200 + 3000 


60 °8 


164 200 + 480 


4+ 300 


164 680 + 1200 


15 200 fs) 4°50 


16 166 080 4 165 880 + 200 
17 a2 fs) 4°8 
18 166 112 4 166 080 + 32 


Perhaps the most detailed psychological investigation of working 
memory comes from two studies of German calculating prodigy Riidiger 
Gamm. Gamm is able to calculate the 9th powers and the 5th roots with great 
accuracy, and find the quotient of two primes to 60 decimal places. Even 
more extraordinary is that he started training for these feats when he was 20 


years old. Before then, his mathematical abilities had been unexceptional. 


Long-Term Working Memory. 


As we have seen, one of the barriers to mental calculation is the limited 
capacity of working memory. Many exceptional calculators use and invent 
algorithms that minimize the load on working memory. It has also been 
suggested that one of the consequences of expertise is the ability to exploit 
the unlimited capacity of long-term working memory (LTWM) in the service 
of the current task (Ericsson & Kintsch, 1995). It is as if experts “develop an 
ability to use long-term episodic memory to maintain task-relevant materials, 
rather as computers extend the capacity of RAM by using swap space on the 
hard drive to create a larger ‘virtual memory’” (Butterworth, 2001, p. 12). 
Language processing is a more familiar example of prodigious skill after 
years of daily practice enabling retention of information well beyond the span 


of short-term working memory. We can effortlessly retain meaningful 


sentences of 20 words or more, well beyond the span for unrelated words 
(about six) or words not in our language (about three). Several related 
accounts of this phenomenon propose cues in working memory for retrieving 
well-organized, domain-specific information in long-term episodic memory 
(Butterworth, Shallice, & Watson, 1990). 

Mitchell (1907) noticed that there seemed to be two types of working 
memory used in calculation, visual and auditory, depending on how numbers 
were initially learned. Most children learn about calculating by counting 
aloud using the names of numbers, often some years before they understand 
written numerals (Gelman & Gallistel, 1978). There appear to be four main 


stages in the development of counting as an addition strategy: 


Counting all. For 3 + 5, children will count “one, two, three” and then 
“one, two, three, four, five” countables to establish the numerosity of the 
sets to be added, so that two sets will be made visible — for example, 
three fingers on one hand and five fingers on the other. The child will 
then count all the objects. 

Counting on from first. Some children come to realize that it is not 
necessary to count the first addend. They can start with three, and then 
count on another five to get the solution. Using finger counting, the 
child will no longer count out the first set, but start with the word 
“Three,” and then use a hand to count on the second addend: “Four, five, 
Six, seven, eight.” 

Counting on from larger. It is more efficient, and less prone to error, 
when the smaller of the two addends is counted. The child now selects 
the larger number to start with: “Five,” and then carries on “Six, seven, 
eight” (Butterworth, 2005). 


Retrieving from memory and decomposition. The child has 
previously calculated 3+5 and can recall the solution. For larger 
addition, such as 6 + 7, the child may be able to recall the double, 6 + 6, 
and then add on the extra 1. However, this requires understanding a 
rather abstract principle, namely, that ifb=a+n,thena+b=a+ta+n, 


and generalizations thereof. 


However, many calculators report that their early experiences involved 
manipulables. Bidder described how he learned multiplication in the 
following way: “I used to arrange [peas, marbles, or shot] into squares, of 8 
on each side, and then on counting them throughout, I found that the whole 
number amounted to 64” (quoted by Smith, 1983, p. 212). It is probable that 
Bidder, like others who had early experience with manipulables, also used a 
kind of visual coding. Salo Finkelstein (born 1896), who seemed to have had 
a standard Polish mathematical education, without showing early signs of 
exceptionality, calculated by visualizing numbers on a freshly washed 
blackboard. His calculation ability seemed not to have equaled many other 
prodigies in terms of time or accuracy, but his ability to memorize numbers 
did. He was able to remember numbers up to about 28 digits following a one- 
second visual exposure; for 39 digits he needed a four-second exposure. He 
was adept at repeating in either direction with equal accuracy, which 
traditionally suggests a visual memory. However, he also used a wide variety 
of associations for substrings to help him, including numerical facts, such as 
the fact that 1.41 is the square root of 2, 2,592,000 is the number of seconds 
in a month, 10,592 is a familiar telephone number, and 2595 is the number of 


paragraphs of Spinoza’s ethics (Smith, 1983, Chapter 33). 


Dehaene and colleagues (Dehaene & Cohen, 1995) have proposed that 


multi-digit arithmetic of the sort carried out by calculators depends on 
visualizing the digits on a kind of mental blackboard. There is some evidence 
that neurological damage can lead to deficits in spatial cognition, which can 
lead to a kind of spatial “acalculia,” where the patient has difficulty in 
maintaining the digits in columns accurately (Hécaen, Angelergues, & 
Houillier, 1961). 

In the case of Gamm, it was possible to identify the brain areas active 
during calculation, and hence whether verbal or visual areas were active. It 
turned out that computation compared to retrieval of memorized number facts 
in both Gamm and the controls activated an extensive visual processing 
system bilaterally. According to Pesenti and colleagues, this finding suggests 
that “during complex calculation, numbers are held and manipulated onto a 
visual type of short term representational medium” (Pesenti et al., 2001, 
p. 103). This finding contrasts with the more usual claim that “sub-vocal 
rehearsal is ... required for mental arithmetic” (Logie, Gilhooly, & Wynn, 
1994, p. 405), but it would explain how it is possible for brain damage to 
reduce digit-span to two and yet allow the patient to reliably add two orally 


presented three-digit numbers (Butterworth et al., 1996). 


Domain-Specificity in Memory. 
Gamm had a forward span of 11 digits (controls 7.2, SD = 0.8) and 12 digits 


backwards (controls 5.8, SD = 0.8), whereas his letter span was in the normal 


range (Pesenti, Seron, Samson, & Duroux, 1999). Henri Mondeux (1826-61), 


a famous nineteenth-century calculator, was described by a contemporary as 


never having learned anything besides arithmetic: “Facts, dates, places, pass 


before his brain as before a mirror without leaving a trace” (quoted by Smith, 
1983, p. 294). 

Long-term working memory (Ericsson & Kintsch, 1995), deployed by 
experts, is specific to the domain of expertise; thus, the musician, the chess 
master, and the waiter could be normal on, for example, digit-span, but 
exceptional in their area of expertise: musicians can recall tunes after a single 
hearing, chess masters can recall positions after a single tachistoscopic 
presentation as well as the whole game that they have just played, and expert 
waiters can keep in mind the precise orders for up to 20 people without 
writing them down (at least until the customer has paid) — the “Zeigamik 
Effect.” Experts develop this kind of LTWM (Ericsson & Kintsch, 1995) and 
as Ericsson and Charness note, “Exceptional memory is nearly always 


restricted to one type of material” (Ericsson & Charness, 1994, p. 735). 


Management and Strengthening Memories. 


Solving even a simple arithmetical problem can be broken down into separate 
components, which will include retrieving arithmetical facts from memory, 
retrieving procedures for calculating (such as borrowing and carrying), 
understanding the arithmetical concepts demanded by the problem, and 
creating a hierarchical set of goals and sub-goals appropriate for reaching the 
solution. Charness and Campbell have shown that, in learning a new 
algorithm for multiplying double-digit numbers, the memory elements are 
strengthened by practice, but there is a larger effect from the overall approach 
to the problem, particularly, from “increased efficiency in managing memory 
and accessing the next step in the procedure” (Charness & Campbell, 1988, 
p. 127), 


Convergent evidence for the compositionality in the brain of 
arithmetical tasks comes from neurological patients, whose arithmetical 
abilities can be selectively affected in very specific ways: the memories for 
facts alone can be lost, indeed the memories for facts from each of the four 
arithmetical operations can be selectively impaired (Cipolotti & van 


Harskamp, 2001; van Harskamp & Cipolotti, 2001); arithmetical procedures 


can be lost from memory (Girelli & Delazer, 1996; McCloskey & 


Caramazza, 1987); and the ability to apply arithmetical principles to 
problems can be selectively spared or affected (Delazer & Benke, 1997; 
Hittmair-Delazer, Semenza, & Denes, 1994). 

Perhaps the greatest of recent calculators, Wim Klein acquired the 
multiplication tables to 100 * 100 from experience gained by factoring. 
However, he did set out to memorize the table of logarithms up to 150. This 
training enabled to him to achieve the world record in extracting roots. He 
could extract the 13th root of a 100-digit number in under two minutes by 
using a method that requires taking logarithms of the left-most group of 
numbers. 

Aitken, similarly, had an enormous store of number facts. For him the 
year 1961 evoked the thoughts 37 ¢ 53, 442 + 52, and 407 + 192. He could also 
recite the first 100 decimal places of m (Hunter, 1962). 

Like other calculating prodigies, Gamm taught himself a vast range of 
number facts. Most of us know our multiplication tables, and perhaps 50 


simple additions (Ashcraft, 1995), but Gamm has learned tables of squares, 


cubes, roots, and so forth. Most of us know a few procedures for working out 
problems that we cannot retrieve from memory, whereas Gamm has an 
enormous store of procedures and shortcuts, some of which he has learned 


from books, others he has worked out for himself. Gamm, again like other 


experts, is able to solve multi-step problems very quickly and accurately. To 
solve 68 * 76 takes seven steps and six intermediate results. After some 
practice with the task, Gamm was taking around five seconds per problem 
with a high degree of accuracy. (Two-digit squares, by contrast, took him just 


over a second because they were simply retrieved from memory.) 


Motivation and Instruction 


Zeal and Inclination 


Most exceptional calculators seem to have been obsessed with numbers from 
the time they began to count. Jedediah Buxton kept a record of all the free 
drinks he received from demonstrating his calculating prowess. Thomas 
Fuller counted the hairs in a cow’s tail, and Arthur Griffiths (1880-1911) 
kept track of the grains of corn he fed to his chickens: 42,173 over three years 
(Smith, 1983, p. 277). Srinivasa Ramanujan (1887-1920), a prodigious 
calculator and, according to G. H. Hardy (1969), a natural mathematical 
genius in the class of Euler or Gauss, would work at mathematics in the 
mornings before work, often having stayed up all night working on problems. 

Calculators from an early age develop a kind of intimacy with numbers. 
When Bidder was learning to count to 100, the numbers became “as it were, 
my friends, and I knew all their friends and acquaintances” (Smith, 1983, 
p. 5). 

Klein told Smith that “Numbers are friends for me, more or less. It 
doesn’t mean the same for you, does it, 3,844? For you it’s just a three and an 
eight and a four and a four. But I say, ‘Hi, 62 squared.’” In a famous story, 
Hardy visited Ramanujan in hospital and mentioned that the taxi in which he 
had come was number 1729, “A rather dull number.” “No, Hardy! It is a very 
interesting number. It is the smallest number expressible as the sum of two 
cubes in two different ways” (C. P. Snow in his introduction to Hardy, 1969). 

In some cases, there is an incident that awakens the interest. For Aitken, 
a teacher “chanced to say that you can use the factorization to square a 
number: a? + b* = (a + b)(a — b) + b?. Suppose you had 47 — that was his 
example — he said you could take b as 3. So (a + b) is 50 and (a — b) is 44, 
which you can multiply together to give 2200. Then the square of b is 9, and 


so, boys, he said, 47 squared is 2209. Well, from that moment, that was the 
light, and I never went back” (Hunter, 1962, p. 252). 


In the case of Gamm, he said that at school he was “very bad at 
arithmetic” because the teachers never explained the concepts in ways he 
could understand. As a result he lost interest in mathematics until about the 
age of 20, when he came across an algorithm for calendrical calculation. He 
practiced it for fun, and then entered for a TV competition where he could 


win bets by solving various calculations. 


The Role of Practice — 10,000 Hours 


The highest level of expertise in violinists studied by Ericsson, Krampe, and 
Tesch-Romer (1993) requires 10,000 hours of practice (by the age of 20 
years). In general, the level of expert performance was related directly 
(monotonically) to the amount of practice. Similarly, expert calculators spend 
a great deal of time learning numerical facts and procedures, though the exact 
amount has never been properly quantified. In preparation for the TV 
program on which he appeared, Gamm started to train up to four hours a day, 
learning number facts and calculation procedures. He now performs 
professionally. His expertise is rare enough to be a cause of wonder (the 
usual definition of prodigy). 

Some of the best evidence for the pure effects of practice comes from an 
experiment carried out by Binet in which he compared the performance of 
two professional calculators, Inaudi and Diamandi, with cashiers from the 
Bon Marché department store in Paris, who had had 14 years’ experience of 
calculating (there were of course no mechanical calculators available in the 
1890s), but who, presumably, showed no special early gift for mathematics. 
He compared how long it took them to carry out multi-digit multiplications. 
Although the timing was about as accurate as it could have been without 
voice-activated relays, it is far from clear that the conditions were the same 
for each participant, and the different participants were mostly given different 
problems to solve. However, they were given one identical 3-digit * 3-digit 


and one 4-digit * 4-digit problem. For these stimuli, the best cashier was 


better than either calculator. As can be seen in Jable 32.2 at least one cashier 


was better than the professionals, but all were better than Binet’s students. 


Table 32.2 Timed calculations by theatrical calculators and from the best 


cashier at Bon Marché, a department store in Paris (from Binet, 1894) 


638 x 823 7286 Xx 5397 
Inaudi 6.4 sec 21 sec 
Diamandi 56 sec 2 min 7 sec 


Best cashier 4 sec 13 sec 


Education 


Ericsson and colleagues have stressed the importance in reaching high levels 
of expertise of “optimal environments” for children and cite examples of 
parents who have designed such environments irrespective of objective 
evidence for innate talent in the children (Ericsson & Charness, 1994). One 
can think of the Polgar sisters in chess, the Williams sisters in tennis, Tiger 
Woods in golf, and so on; Mozart grew up in a musical household, and 
Picasso’s father was himself a painter. 

Vito Mangiamele (1827-59) was the son of a shepherd who was unable 
to give the boy any instruction. According to Scripture, “By chance it was 
discovered that by methods peculiar to himself, he resolved problems that 
seemed at the first view to require extended mathematical knowledge,” for 
example, “What satisfies the condition that its cube plus five times its square 
is equal to 42 times itself increased by 40?” (x? + 5x* — 42x — 40 = 0). He 
found the answer to this (5) in less than a minute when he was 10 years old 
(1891, p. 17). 

Zacharias Dase (1824-61) was an extraordinary calculator who, for a 
time, assisted Gauss in calculating tables. One distinguished mathematician 
credited him with “extreme stupidity,” a view that seemed to be held by his 
mathematician collaborators. He knew no geometry and never mastered a 
word of another language. “He had one ability not present to such a great 
degree in other ready reckoners. He could distinguish some thirty objects of a 
similar nature in a single moment as easily as other people can recognize 


three or four. The rapidity with which he would name the number of sheep in 


a herd, or books in book-case, or window-panes in a large house, was even 
more remarkable than the accuracy of his mental calculations” (Scripture, 
1891, p. 20). According to Mitchell, he “could count some thirty objects at a 
glance” (1907, p. 142), though it is not clear what this had to do with his 


calculating prowess. 


Abacus Training 


The abacus has been used in Asia since 1200 for rapid precise calculation, 
and may have emerged from earlier Roman counting boards, which bear a 


similar structure (Menninger, 1969, p. 204). 


Some of the most extraordinary feats of calculation can be seen in 
abacus competitions in the Far East, especially in Japan and Taiwan. Here 
children and adults demonstrate astonishing feats of calculation, adding a 
series of numbers presented several per second (speed depends on the 
competition level). Such performances would surely count as prodigious — a 
cause of wonder. As far as one can tell, there is nothing special about the 
children who compete apart from abacus training and the willingness to do 
what is necessary to be outstanding. Typically, the competitors do not need to 
use an actual physical abacus, with early learners moving their fingers as if 
manipulating the beads (www.youtube.com/watch?v=FgFEckm-rRA), but 
more experienced calculators report using a “mental abacus.” 

Children are subjected to extensive abacus training in Japan, China, 
Taiwan, India, and many other countries, usually in after-school classes. This 
can involve many years and hundreds of hours of deliberate practice, often 
directed toward success in competitions, such as “flash anzan,” where 
competitors have to add numbers presented at a rate where it is scarcely 
possible to read them, let alone remember and manipulate them. For example, 
the 2012 world champion, Naofumi Ogasawara, is able to add correctly 15 
four-digit numbers presented at 0.4 sec each (www.youtube.com/watch? 


v=JawFOcv50Lk). As far as I know, there has been no scientific study of how 


these amazing feats are achieved, either in terms of cognitive theory or 
neuroimaging. 

One fascinating feature of the ability to use a mental abacus is that you 
can carry out another task in parallel provided it does not involve numbers. 
So in one video clip (www.youtube.com/watch?v= vGMsVirYKs), the girls 
are playing a challenging language game while adding a sequence of rapidly 
presented numbers. This suggests that calculation is a separable mental 
process from other aspects of cognition. We will see later that the neural basis 
for abacus use, as well as other types of complex calculation, is separate from 
other types of reasoning and memory. 

This is consistent with a recent study by Frank and Barner that showed 
that practiced users of a mental abacus were relatively insensitive to verbal 


interference (Frank & Barner, 2011). However, Frank, Barner, and colleagues 


found in a three-year longitudinal study, that 100 hours of abacus training 
benefited children who already had good visuospatial working memory. They 
improved their computational accuracy and fluency, but their basic cognitive 


abilities were unchanged (Barner et al., 2016). 


Genetics 


Galton’s account of the parents and offspring of men of eminence did not 
examine the potential social and educational effects of growing up in a 
talented and well-connected family. Of course, there will be cases like 
D’Alembert, and those described earlier, who have achieved eminence 
despite an apparently unhelpful upbringing. 

Genetics researchers support the idea of an innate domain-specific 
system for at least simple mathematics. A recent twin study of mathematical 
abilities showed that the concordance rates were 0.73 for monozygotic and 
0.56 for dizygotic pairs (Alarcon, Defries, Gillis Light, & Pennington, 1997). 
Looking at the selective deficit of mathematical ability, dyscalculia, of the 
dyscalculic probands, 58 percent of monozygotic co-twins and 39 percent of 
dizygotic co-twins were also dyscalculic. In a family study, it was found that 
approximately half of all siblings of children with dyscalculia are also 
dyscalculic, with a risk five to ten times greater than for the general 
population (Shalev & Gross-Tsur, 2001). 

Another line of research has attempted to assess whether sex-linked 
characteristics contribute to mathematical expertise. Benbow and colleagues 
have found in a host of studies a significant advantage for talented 12- to 13- 
year-old boys over girls at the upper end of the ability range, as measured by 
SAT-M (Scholastic Aptitude Test - Mathematics), whereas SAT-V (Verbal) 
showed no comparable difference (see Benbow, 1988, for a review). Benbow 
argues that the sex difference cannot be explained in terms of 


“environmental” hypotheses to do with attitudes, confidence, or teaching. She 


argues rather that a combination of biological differences between the sexes 
is the cause, in particular a more bilateral neural representation of cognitive 
functions in the female brain (see the following section), 

The differences between boys and girls in SAT-M performance appear 
to follow from the much larger variance in boys’ scores, which would allow 
reliable differences at the top end of the range even if the mean score for girls 
were higher than for boys (Becker & Hedges, 1988). When one looks at the 
means, girls in England easily outperform boys in all subjects at all ages. 
There is one exception to this general rule: mathematics. Girls are only just 
outperforming boys (DfES, 2002). 

On the other hand, Geary reviewed a wide range of industrialized 
countries and showed that boys, on average, still outperform girls in 
mathematical problem-solving (Geary, 1996). However, even in the USA at 
17 years the average difference between boys and girls is still only 1 percent. 
The most recent cross-national comparisons using the same tests in all 
countries, the Third International Maths and Science Survey (TIMSS) (Keys, 
Harris, & Fernandes, 1996), reinforces the overall picture that in most 
countries, including the USA, there is no statistical difference in the means, 
though there are enormous differences among countries, suggesting that 
educational and cultural factors are vastly more important than gender in the 


acquisition of mathematical skills. 


Brain Systems for Mathematical Expertise 


There is now extensive evidence that routine numerical tasks involve a 
frontoparietal network (Pesenti, Thioux, Seron, & De Volder, 2000), where 
the parietal components, especially the left intraparietal sulcus, are relatively 
specialized for numbers (Dehaene, Piazza, Pinel, & Cohen, 2003). It is 
certainly the case that damage to the left parietal lobe can severely affect 
calculation (Cipolotti & van Harskamp, 2001), though almost nothing is 
known about its effect on other mathematical domains. 

More complex calculation in relatively non-expert participants 
established that the neural basis of simple retrieval (e.g. 3 * 4 = ?), relative to 
a reading control, “engaged a left parieto-precentral circuit representing a 
developmental trace of a finger-counting representation that mediates, by 
extension, the numerical knowledge in adults,” plus a naming network 
including the left anterior insula and the right cerebellar cortex (Zago et al., 
2001, p. 325). On the other hand, complex computation (e.g. 32 * 24) 
engaged, additionally, a left parieto-superior frontal network for holding 
multi-digit numbers in visuospatial working memory along with bilateral 
inferior temporal gyri, which is implicated in visual mental imagery. 
Correlated activity in the left intraparietal sulcus and the precentral gyrus 
may reflect the involvement of a finger movement representation network in 
the calculation process. This is not to say that these skilled adults are 
counting on their fingers, but it may be that the childhood use of fingers in 
learning to calculate somehow creates the neural substrate for later 


acquisition of numerical knowledge (Butterworth, 1999). 


There have been very few studies of the brain systems of expert 
calculators. Benbow, O’Boyle, and colleagues (e.g. Alexander, O’Boyle, & 
Benbow, 1996; O’Boyle, Benbow, & Alexander, 1995; O’Boyle, Gill, 
Benbow, & Alexander, 1994; Singh & O’Boyle, 2004) have investigated 


mathematically gifted children and adolescents, with special reference to 


gender and brain organization. In general, they have found more right- 
hemisphere involvement in a range of tasks, though, curiously, mathematical 
tasks themselves have not been studied. Pesenti and colleagues have 
published data on the brain of an expert calculator carrying out mathematical 
tasks (Pesenti et al., 2001). 

In a functional neuroimaging study, Pesenti and colleagues found that 
the prodigy Gamm’s calculation processes recruited the same neural network 
as previously observed for both simple and complex calculation (Zago et al., 
2001), plus a system of brain areas implicated in episodic memory, including 
right medial frontal and parahippocampal gyri, whereas those of control 


subjects did not (Butterworth, 2001; Pesenti et al., 2001). Functional brain 


imaging has established that speech-based working-memory storage, of the 
kind that supports standard digit-span tasks, involves the perisylvian 
language areas (Paulesu, Frith, & Frackowiak, 1993), but Gamm’s activations 
here are quite different. As noted above, it has been suggested that experts 
develop a way of exploiting the unlimited storage capacity of long-term 
memory to maintain task-relevant information, such as the sequence of steps 
and intermediate results needed for complex calculation, whereas the rest of 
us still rely on the very limited span of working memory (Ericsson & 
Kintsch, 1995). Gamm’s activations are consistent with his having developed 
LTWM for arithmetical calculations (Butterworth, 2001). 


It seems clear from a small number of studies, that professional 


mathematicians use the same brain circuits as non-mathematicians, but 
modify them. For example, one study found a small region in left 
intraparietal sulcus — the “classical” number area — that shows greater gray 


matter density in the brains of mathematicians than matched controls — other 


academics (Aydin et al., 2007). Strikingly, the increase in gray matter density 
in this region — but nowhere else in the brain — was correlated with the 
number of years of professional mathematical practice. 

In a functional MRI study, Amalric and Dehaene found that the brain 
network activated in professional mathematicians when making judgments 
was very similar to the brain network that the rest of us use for everyday 
arithmetic, specifically the frontoparietal network. The temporal lobe site for 
general knowledge semantics and the language areas were not activated. 
Thus, mathematicians using their expertise deploy these same brain regions 
as non-mathematicians doing simpler mathematical tasks, but use them more 
actively, and it has been suggested that this brain network, which has an 
innate basis, is “recycled” to cope with the demands of culturally acquired 
competencies (Amalric & Dehaene, 2016). 

There is some evidence that abacus training, unsurprisingly, sets up a 
different brain network for calculation. For example, individuals with a 
mental abacus in mental arithmetic tasks activate motor regions of the brain, 
especially dorsal premotor cortex bilaterally, as well as the classical number 
regions in the inferior and superior parietal lobes. Tanaka and colleagues 
present a fascinating recent study of “abacus-based acalculia,” in which an 
expert abacus user reported a loss of her mental abacus following a stroke. 
She had suffered an extensive right frontoparietal infarct, including damage 
to the right dorsal premotor cortex and the right inferior parietal lobe (Tanaka 


et al., 2012). Nevertheless, one month after her stroke she was in the normal 


range on a variety of calculation and number memory tasks, though 
presumably not at her excellent pre-morbid level when she was able to use 
her mental abacus. At 6 months post-stroke, she was still unable to image an 
abacus, but at 13 months, she had recovered the image. So how had her brain 
activity changed from 6 to 13 months? This is what Tanaka and colleagues 


conclude: 


The results of the present study show that brain activity during mental 
calculation at 13 months after her stroke was observed more in an area 
implicated in visuospatial working memory ... whereas at 6 months 
after her stroke, brain activity was more predominant in the left 
hemisphere in areas related to verbal working memory ... Brain activity 
at 13 months after her stroke was observed in the left SPL, whereas that 
at 6 months after her stroke was observed in Broca’s area and the left 
DLPFC [dorsolateral prefrontal cortex] and IPL [inferior parietal lobe]. 
This shift of region-specific brain activities is consistent with her 
subjective report that she was able to shift her calculation strategy from 
a verbal to a visuospatial strategy according to the level of her recovery 
from the stroke. Strikingly, at 13 months, a picture of an abacus, but not 
a face, interfered with digit memory, confirming her use of the mental 


abacus. 


(Tanaka et al., 2012, p. 8) 


This suggests that perhaps the right hemisphere is more involved in 
calculations carried out by abacus experts than by typical calculators. It is 
worth noting, however, that this woman was left-handed and perhaps the 
lateralization of her brain reflected this; nevertheless, the brain activations in 


the left parietal lobe seemed to reflect the typical organization of numerical 


functions, along with the fact that her language was unimpaired and 
calculation is normally lateralized on the same side as language (Semenza et 
al., 2006). 


Conclusions 


Our starting point was Galton’s tripartite theory of eminence: capacity, zeal, 
and the ability to do a very great deal of hard work. 

Starting with capacity, it is clear that individuals with exceptional 
mathematical, and especially calculating, ability show enormous variety of 
cognitive abilities. Some are highly intelligent, others averagely intelligent, 
yet others are classed by their peers (before standardized IQ testing) as 
stupid. So the kind of general intellectual capacity supposed by Galton does 
not seem to apply here. Nor does our survey support Gardner’s (1983) idea of 
a distinct “logical-mathematical” intelligence, since many prodigies seem no 
better than average, and indeed many are much worse than average, in 
reasoning. 

Zeal seems to be a characteristic common to all the prodigies described 
here. They are obsessed with numbers, treat them as familiar friends, and 
actively seek closer acquaintance with them. 

Is their exceptional ability confined to mathematics? Whereas some 
seem to excel only in calculation, others have shown eminence in fields other 
than mathematics. 

Finally, are exceptional calculators born or made? There is ample 
evidence for zeal and hard work, and it may be that we are born with 
dispositions toward them. Charles Darwin, in a letter to Galton, wrote “I have 
always maintained that excepting fools, men did not differ much in intellect, 
only in zeal and hard work; I still think this an eminently important 


difference” (quoted by Ericsson & Charness, 1994). It may also be the case 


that some of us are born with a disposition to enjoy or even be obsessed with 
an orderly domain like mathematics. Although there is now ample evidence 
for a specific congenital disability for arithmetic (Butterworth, Varma, & 


Laurillard, 2011), there is no evidence at the moment for differences in innate 


specific capacities that lead to mathematical expertise. 
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Introduction 


In order to communicate and express meaning in both their first (L1) and 
second language (L2), people need a large vocabulary. Thus, knowledge of 
vocabulary is fundamental to all language use and must be learned in some 
manner in order for learners to become communicative in a new language. 
However, the lexicons of most languages are extensive with English having 
the largest vocabulary of any known language (Schmitt & Marsden, 2006) 
and a vast number of loanwords from other languages. Therefore, learning 
vocabulary is an essential part of mastering a second language as it is likely 
to be the biggest hurdle for successful communication. 

There is plenty of evidence pointing to the importance of vocabulary in 
language use. It has been acknowledged that vocabulary knowledge is a good 
predictor of general proficiency in a second language with strong 
relationships between vocabulary measures and the ability to read, write, 


listen, and speak (e.g. Laufer & Goldstein, 2004; Stahr, 2008; Zimmerman, 


2004). Generally speaking, the more words learners know, the more 
proficient they become, and the better they are likely to perform whatever the 
skill. Considering the multitude of factors that could affect learners’ 
performance (e.g. learner motivation, background knowledge, familiarity 
with test task), it is striking that the single factor of vocabulary accounts for 
such a large percentage of the variation in the performance of the four skills 
(e.g. 40-62 percent; Alderson, 2005). In short, vocabulary knowledge is 


clearly a major contributor to success in language performance. 


Since knowledge of vocabulary is essential to function in English, it is 
important to know how much vocabulary is needed to accomplish specific 
tasks (e.g. reading a book in English, listening to a lecture, etc.). This is 
because mastery of the complete lexicon of English is beyond not only 
second language learners, but also native speakers, as English is estimated to 
have 54,000 word families (Goulden, Nation, & Read, 1990). A word family 
includes a base word, and its inflected and derived forms (Hirsh & Nation, 
1992); for example, the word family “nation” has 26 members and national, 
nationalism, nationalist, nationalize, and nationhood are just a few of them. 
This roughly corresponds to a dictionary entry, and few native speakers 
would be expected to know every word in a dictionary. It is estimated that 
well-educated native speakers know around 20,000 word families, excluding 
proper names and derived forms (Goulden et al., 1990; Zechmeister, Chronis, 
Cull, D’Anna, & Healy, 1995). However, reaching this vocabulary size 
would be a daunting, and probably unmanageable task for most second 
language learners. Fortunately, second language learners do not need to 
achieve native-like vocabulary sizes in order to use English well. Instead, 
they require much smaller amounts of vocabulary in order to be functional in 
specific communicative contexts. 

The primary focus of this chapter is to describe the L2 vocabulary 
expertise necessary to be functional in a variety of situations, such as reading 
different types of books, watching films, and taking part in a conversation or 
a seminar. We believe there are three key requirements necessary to achieve 
this expertise. The first requirement is learning enough words to 
communicate successfully in English. Without a large enough vocabulary, 
neither language production nor language comprehension is possible. The 


second requirement is learning various aspects of word knowledge, which 


involves knowing enough information about each word. The third 
requirement is learning how words combine to form longer phrases. These 
word combinations are as important as individual words, and thus are crucial 
for reaching high levels of proficiency. 

In the following sections, we present a more detailed account of lexical 
coverage and vocabulary size, vocabulary depth, and formulaic language, and 


highlight their importance for gaining expertise in L2 vocabulary. 


Lexical Coverage 


How Much Coverage Is Needed for Successful Comprehension? 


Before one can calculate the vocabulary size necessary to function in English, 
it is first necessary to determine the percentage of words in a text that a 
person needs to know in order to comprehend that text (lexical coverage). 
This is likely to vary between reading and listening for several reasons. 
Readers can refer back to lexical items in a text to decode the message, 
whereas this is not possible for listeners (Reves & Levine, 1988). Listeners 
often struggle to recognize individual spoken lexical forms correctly, while 
this is not the case in reading. Yet another difference is that spoken discourse 
tends to be lexically less dense (i.e. containing a smaller proportion of 
content words such as nouns, verbs, adjectives, and some adverbs) than 
written text (Carter, 2012) and contains many fillers and repetitions (Rost, 
2002). Thus, the lexical coverage required is different for written versus oral 


modes. 


Reading 


Most research has been conducted in relation to reading, with fewer studies 
focusing on coverage in listening. Hu and Nation (2000) was one of the first 
studies to suggest lexical coverage requirements in reading. The authors 
examined the relationship between coverage and reading comprehension in a 
fiction text. Four different coverage degrees were created (80%, 90%, 95%, 
and 100%) by replacing some real words in the target text with non-words 
(e.g. rane, sorant, yeard), and then testing the resultant L2 comprehension. 
Results showed that no leamer achieved adequate comprehension 
(conceptualized as the score that most learners in the 100% coverage group 
received) at 80%, while only a few did at 90% and 95%. Adequate 
comprehension was achieved somewhere between 95% and 100%, which led 
the authors to suggest that it takes 98-99% coverage to allow unassisted 
reading for pleasure. 

Likewise, Schmitt, Jiang, and Grabe’s study (2011) explored whether 
more vocabulary would lead to more comprehension. The participants 
completed a vocabulary checklist test based on words drawn from two 
academic texts, then read the texts, and completed a reading comprehension 
test for each. Results demonstrate a linear relationship between the 
percentage of vocabulary known and text comprehension, which indicates 
that the coverage level required depends on the degree of comprehension 
aimed for. If 60% comprehension is considered adequate, then 95% coverage 
is necessary. If the goal is 70% comprehension, then 98-99% coverage is 
required. If the aim is 75% comprehension, then the data suggest 100% 


coverage is needed. This indicates that factors other than coverage contribute 


to full comprehension (e.g. background knowledge, text features, ability to 
infer from context, etc.). 

In a similar study, Laufer and Ravenhorst-Kalovski (2010) looked at the 
relationship between second language learners’ vocabulary level, the lexical 
coverage that their vocabulary provided, and their reading comprehension. 
Vocabulary size was measured through the Vocabulary Levels Test (Schmitt, 


Schmitt, & Clapham, 2001), lexical coverage by the then newest version of 


Vocabulary Profile (Cobb, n.d., available at www.lextutor.ca), and reading 


comprehension by a standardized national test based on academic texts. Two 
coverage threshold points were established: 98% as optimal (predicting 
independent reading) and 95% as minimal (which characterizes assisted 
reading). 

Based on the results of the above studies, the consensus is that 98% is 
the coverage figure which is most appropriate for most reading purposes. 
However, this 98% figure is derived mainly from a small number of 
academic texts, of limited length, which were read without access to 
resources. Therefore, it is yet to be determined whether non-academic texts 
would require more or less coverage in order to be adequately understood, 
and whether longer passages or greater access to resources would alter the 


coverage figure required. 


Listening 


There has been less research on lexical coverage in the oral mode. Bonk 
(2000) was the first to examine lexical coverage in relation to L2 listening 
comprehension. In his study, the subjects listened to four short passages with 
increasing percentages of low-frequency words. Dictation and written free- 
recall tests were used to measure the participants’ lexical knowledge and 
comprehension. Results revealed that higher dictation scores led to better 
listening comprehension. Learners with lexical coverage scores below 90% 
showed poor comprehension, whereas learners with lexical coverage scores 
over 95% showed good comprehension (i.e. achieved scores of 3 and 4 ona 
four-point scale that was used to rate the answers on the free-recall test). 

A more recent study by van Zeeland and Schmitt (2013) directly 
examined the lexical coverage/listening comprehension relationship by 
having their participants listen to four spoken informal narrative passages. 
The coverage of the listening material was manipulated by replacing words 
from the passages by non-words resulting in different percentages of known 
vocabulary, namely 100%, 98%, 95%, and 90%. The best comprehension for 
non-natives came from the 100% passage (9.62 out of 10 maximum), then the 
98% passage (8.22/10), but there was no statistical difference between 95% 
and 90% (7.65/10 and 7.35/10 respectively). However, there was much more 
variability at 90% coverage than at 95% coverage, with some learners being 
able to cope, but others not. This led the authors to conclude that 95% lexical 
coverage was necessary for adequate listening comprehension, because the 
participants achieved reasonable comprehension (7.65/10), but with much 


more consistency than at 90% coverage. 


However, one of the limitations of this study is that it used informal 
narrative stories, and did not indicate whether the same level of coverage was 
required for other kinds of listening contexts (e.g. academic lectures). 
Therefore, while there is initial evidence that 95% lexical coverage may be 
adequate for listening, this figure must be established across a wide range of 
listening contexts, and may well be higher for more complex texts, e.g. 98%. 

Overall, the above research suggests that the higher the level of expected 
comprehension, the higher the vocabulary demands. It also shows that even 
high levels of lexical coverage did not lead to 100% comprehension, which 
suggests that vocabulary is only one aspect of comprehension. Other factors 
(e.g. reader purpose, task goals, and background knowledge) are equally 
crucial and need to be accounted for as well. However, lexical coverage was 
shown to be a key factor. Research points to a lexical coverage of around 
98% as necessary for adequate reading comprehension (Nation, 2006; 
Schmitt, Cobb, Horst, & Schmitt, 2017). In terms of listening, research 
suggests that 95% coverage may be sufficient (van Zeeland & Schmitt, 
2013). However, more difficult listening contexts may require higher lexical 
coverage (Rodgers & Webb, 2011; Schmitt et al., 2017). 


Vocabulary Size: How Much Vocabulary Do 
Learners Need to Know? 


Based on the above lexical coverage targets, it is possible to calculate how 
much vocabulary is necessary to reach those targets. Varying estimates have 
been put forward by different researchers, but they all seem to suggest that 
for everyday conversation purposes (e.g. describing everyday activities, 


telling the time, etc.), learners need around 2,000—3,000 word families, based 


on a 95% coverage requirement (e.g. van Zeeland & Schmitt, 2013). To 
engage in a wider range of more sophisticated conversation topics (e.g. 
politics, studying at university), knowledge of around 6,000—7,000 word 
families is thought necessary, based on a 98% coverage requirement 
(Rodgers & Webb, 2011; Schmitt et al., 2017). In order to read a wide range 


of authentic texts, such as novels or newspapers, one should aim for around 


8,000—9,000 word families, based on a 98% coverage requirement (Laufer & 
Ravenhorst-Kalovski, 2010; Nation, 2006; Schmitt et al., 2017). 


There has also been research on the amount of vocabulary required to 


watch television programs and movies. Rodgers and Webb (2011) analyzed 
288 television episodes and concluded that it took knowledge of the most 
frequent 3,000 word families to reach 95% lexical coverage, and about 
6,000—7,000 word families for 98% coverage. In a similar study, the same 


authors (Webb & Rodgers, 2009) examined the scripts of 318 movies and 


found similar coverage figures. Nevertheless, there was a considerable 
variation among the individual programs and movies regarding the amount of 


vocabulary required. 


It is also important to understand how vocabulary behaves in terms of 
frequency. It is well known that a small number of words occur very 
frequently and cover a large proportion of the running words (the total 
number of words) in a written or spoken text. Conversely, a very large 
number of words occur very rarely and make up the remainder of running 
words in discourse. This is illustrated in Figure 33.1, which shows Nation’s 
(2006) analyses of nine written and spoken corpora, such as novels and 
newspapers, everyday conversation with friends and family, and people 


calling in to radio talk-show programs. 
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Figure 33.1 Frequency and coverage of vocabulary (based on Nation 
2006, in Schmitt & Schmitt, 2014, p. 487). 


Figure 33.1 shows that the first 1,000 word families clearly provide a 
substantial amount of coverage in English, largely due to the extremely high 
frequency and coverage of function words (i.e. words that express 


grammatical relationships with other words within a sentence, such as and, 


of, or). The second 1,000 contribute a much smaller, but still useful, 
percentage of coverage, as do the third 1,000 to a lesser extent. But by the 
fourth 1,000 families, however, the coverage decreases substantially, with a 
maximum of only 3% for 2,000 families (fourth and fifth 1,000). Beyond 
this, the coverage return becomes increasingly small. 

Thus, the most frequent words provide the greatest amount of coverage, 
and this is why learners need to know these words in particular, rather than 
just any words. A useful way of thinking about frequency is by distinguishing 
between high-, mid-, and low-frequency words (Nation, 2006; Schmitt & 
Schmitt, 2014). The first 2,000—3,000 word families have generally been 
accepted as high-frequency vocabulary and with this vocabulary size, learners 
can communicate to some extent. These words are necessary to use English at 
the basic, but still useful, end of the proficiency continuum. 

Despite their usefulness, however, knowledge of these words is not 
sufficient for successful comprehension of a text. To illustrate this point, the 
following example shows what a text looks like to a second language user 
who only knows the most frequent 2,000 words. Words that are not on the 


2,000 list have been replaced by gaps: 


If planting rates are with planting satisfied in each 
and the forests milled at the earliest opportunity, the wood 
supplies could further increase to about 36 million meters 


in the period 2001-2015. 


(Nation, 1990, p. 242) 


It is evident from the above example that someone reading this text would 


know what the topic is, but not exactly what is being stated about it. 


The mid-frequency vocabulary includes word families between the 
2,000-—3,000 and the 8,000—9,000 level, whereas low-frequency vocabulary 
covers words beyond the 9,000 level (Schmitt & Schmitt, 2014). Following 


Nation’s (2001) advice, high-frequency words should be learnt and taught 
explicitly, whereas the mid-frequency vocabulary could be acquired through 
the use of strategies (e.g. guessing from context), extensive reading, and 
watching television programs from the same genres (e.g. Webb, 2011). 
Extensive reading, defined as “learning of vocabulary as the by-product of 
any activity not explicitly geared to vocabulary learning” (Hulstijn, 2001, 
p. 271), is particularly important as it allows incidental vocabulary learning to 
occur. Because of the large number of words in English, they cannot all be 
explicitly taught and the majority of them have to be learned incidentally. 
Even though incidental leaning is a slow process, it can be effective 
provided that learners encounter the new vocabulary enough times within a 
short time span (Pellicer-Sanchez & Schmitt, 2010; Waring & Takaki, 2003; 
Webb, 2007). 

In summary, in order to become proficient in their L2 and achieve a high 
level of comprehension, learners need to know a large number of high- 
frequency and mid-frequency lexical items to operate in authentic 


environments without being handicapped by unknown vocabulary. 


Depth of Knowledge 


Not Just Quantity 


In vocabulary knowledge, as in many other areas of expertise, it is not only 
quantity that matters, but also quality. Having a large vocabulary size is 
essential to function in English, but how well each individual word is known 
is equally crucial. In order to gain expertise in L2 vocabulary, second 
language learners must know both a large number of words and enough 
information about each word. These two dimensions of vocabulary 
knowledge are commonly referred to as vocabulary breadth and vocabulary 
depth (Schmitt, 2010). Anderson and Freebody (1981, p. 93) were the first to 
develop a conceptualization of vocabulary knowledge involving both breadth 
and depth. They defined the former as “the number of words for which the 
person knows at least some of the significant aspects of meaning” and the 
latter as “the quality or depth of understanding.” A third dimension of 
vocabulary knowledge is fluency (or automaticity), which refers to the ability 
to access words in the mental lexicon in a fast manner. This is a somewhat 
neglected dimension of vocabulary knowledge, even though most people 
would agree that knowing a lot about a word is of limited value unless that 
word can be used fluently in communication. Figure 33.2 illustrates the 
metaphor of lexical space by Milton and Fitzpatrick (2014, p. 9), representing 


the three dimensions of word knowledge: breadth, depth, and fluency. Since 
the fluency dimension is rarely included in discussions of vocabulary 
knowledge (as some researchers see fluency as part of depth of knowledge), 


it will not be explored further in this chapter. 
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Figure 33.2 The lexical space (based on Daller, Milton, & Treffers-Daller, 
2007, in Milton & Fitzpatrick, 2014, p. 9). 


Although they are often described and analyzed in isolation, Schmitt 
(2014) argues that the two dimensions of breadth and depth might be 
interrelated. In other words, when one increases, the other increases as well, 
and we would expect someone with a very large vocabulary size to show 
elaborate knowledge about the words they know. This is because a large 
vocabulary size suggests a high level of language exposure and learning 
experience, which is conducive to more detailed and refined word 
knowledge, and vice versa (e.g. Li & Kirby, 2015; Meara, 1996). On the 
other hand, at the onset of language learning, only a limited number of words 
are known in a rather superficial manner. At the most basic level, the 
form—meaning link is established and a particular string of letters (or 
sequence of sounds) becomes associated with a referent. For example, the 
string of letters cat becomes associated with “a four-legged creature that 
purrs.” As vocabulary knowledge develops, two processes tend to occur 
simultaneously: more and more form—meaning links are created (i.e. more 


words become known), and knowledge of the previously acquired 


form—meaning links is developed further and becomes more detailed and 
precise. 

In some cases, however, vocabulary size and depth might not develop in 
parallel (Schmitt, 2014). It is indeed possible to learn a lot about a small 
number of words, and conversely, a little about a large number of words. 
Depending on each learner’s learning goals in L2, they might approach 
vocabulary learning in different ways. If the goal is to learn many words in a 
limited amount of time (e.g. in preparation for a high-stakes proficiency test), 
then acquiring only the form—meaning link of those words might be enough. 
Memorizing wordlists or using flashcards are two popular activities for such 
a purpose (Nation, 2013). On the other hand, if the goal is to focus on a 
specialized set of words that will be particularly relevant and useful to a 
learner (such as in preparation for an academic course in a specific subject), 
then extensive study of the various aspects of those words is required. Some 
relevant activities might be consulting a dictionary of synonyms/antonyms or 
studying the word in context. If the goal is to become an expert in L2 
vocabulary, then substantial exposure to the language will be necessary to 
enable the acquisition both of many words, and of all the different aspects 


involved in knowing each individual word. 


What Does Depth Mean? 


Vocabulary knowledge is a complex and multifaceted construct (Schmitt, 
2010). Although the notion of vocabulary size is straightforward and easy to 
understand, the concept of vocabulary depth is much vaguer. It is not always 
clear what it entails, as evidenced by the various definitions and 
conceptualizations of vocabulary depth in the literature. At the most basic 
level, depth is synonymous with quality or richness of word knowledge and is 
characterized as the counterpart to breadth within a two-dimensional view of 
vocabulary knowledge (Anderson & Freebody, 1981). It has also been 
conceptualized as developmental, and typically measured along a continuum 


from no knowledge at all to full mastery (Paribakht & Wesche, 1997; Read, 


2000; Schmitt & Zimmerman, 2002). For example, Table 33.1 presents a list 


of several proposed stages of word acquisition suggested by Paribakht and 
Wesche (1997, p. 180). 


Table 33.1 Stages of word acquisition. 


Stage 1 The word is not familiar at all 

Stage 2 The word is familiar but the meaning is not known 

Stage 3 A correct synonym or translation is given 

Stage 4 The word is used with semantic appropriateness in a 
sentence 

Stage 5 The word is used with semantic appropriateness and 


grammatical accuracy 


Another way of looking at depth of knowledge is by breaking it down 
into separate elements, which could be described as a components or 
dimensions approach (Nation, 1990, 2001, 2013; Richards, 1976). Following 
Richards’s original idea, Nation (2013) has proposed the most comprehensive 
and complete conceptualization of word knowledge to date, according to 
which knowing a word involves the mastery of nine aspects: spoken form, 
written form, word parts, meaning, concept and referents, associations, 
grammatical functions, collocations, and constraints of use (Table 33.2). Each 
of these aspects has a productive and receptive variant, and is grouped under 


one of three main aspects of word knowledge: form, meaning, and use. 


Table 33.2 Conceptualization of word knowledge: form, meaning, and use 
FORM Spoken [R] | What does the word sound like? 
[P] How is the word pronounced? 
Written [R] | What does the word look like? 
[P] How is the word written and spelled? 


Word parts [R] | What parts are recognizable in this 
word? 


[P] What word parts are needed to 
express the meaning? 


MEANING _ Form and [R] | What meaning does this word form 
meaning signal? 


[P] What word form can be used to 
express this meaning? 


Concept and [R] What is included in the concept? 


referents 
[P] What items can the concept refer to? 


Associations [R] What other words does this make us 
think of? 


[P] What other words could we use 
instead of this one? 


USE Grammatical [R] In what patterns does the word 
functions occur? 


[P] In what patterns must we use this 
word? 


Collocations [R] | What words or types of words occur 
with this one? 


[P] What words or types of words must 
we use with this one? 


Constraints [R] Where, when, and how often would 
on use we expect to meet this word? 


(register, 
frequency...) [P] | Where, when, and how often can we 
use this word? 


Note: R = receptive knowledge; P = productive knowledge 


The above approaches highlight the ways vocabulary depth has been 
conceptualized in relation to individual words. However, vocabulary mastery 
can also be defined in terms of the overall mental lexicon, or network of 


interconnected words (e.g. Meara, 1996; Milton & Fitzpatrick, 2014). 


According to this view, having a great depth of knowledge means having 
strong interconnections between words (i.e. a highly organized mental 
lexicon). The more developed the lexical network, the quicker it becomes to 
recognize and retrieve words. Figure 33.3, taken from Meara and Wolter 
(2004, p. 89), illustrates the network view of vocabulary knowledge. 


Vocabulary size and organisation 


Figure 33.3 A network way of looking at vocabulary (Meara & Wolter, 
2004, p. 89). 


From a network perspective, we would assume that the ability to use 
words is strongly related to how well the various words (and their various 
word knowledge components) are known and interrelated. To further explore 
what being proficient in L2 vocabulary really means, the next section will 
examine the various components of word knowledge (Table 33.2) in more 


detail. 


Knowledge of Form 


The first main aspect of word knowledge is FORM. In the minds of all 
speakers, concepts and referents are attached to labels and without knowing 
the labels, the concepts cannot be expressed. Due to their previous experience 
of the world in their L1, L2 learers will already be familiar with various 
objects and notions and will know how to express them in their native 
language (e.g. in French, chat is the form attached to “a four-legged animal 
that purrs”). Learning a new word in the L2 is first and foremost a matter of 
relabeling (i.e. of associating a new form to a referent). It is worth 
mentioning, however, that not all semantic concepts are transferable between 
languages and this is especially true for those that do not directly refer to 
physical objects (e.g. the concept of Schadenfreude — pleasure derived from 
the misfortune of others — in German). 

The new L2 form of a previously known L1 concept or referent needs to 
be acquired in both its written and spoken forms. The spelling needs to be 
known so the word can be recognized at a receptive level (when reading) and 
used at a productive level (when writing). The pronunciation of the word 
must be known as well in order for learners to recognize it when heard and to 
produce it in speech. Research suggests that learning the spoken form of a 


word is more difficult for second language learners than learning the written 


form (Milton, Alexiou, & Mattheoudakis, 2014). Also, because words may be 
pronounced differently when used in isolation in comparison to continuous 
speech, some L2 learners may fail to recognize them (van Zeeland, 2014). 
This is where the setting of L2 acquisition becomes crucial. While naturalistic 


settings (i.e. direct immersion in the L2 environment) feature words in natural 


speech, classroom settings are characterized by words as individual written 
units (Nation, 2013), thereby hindering the recognition and production of 
spoken word forms. 

Because of the spontaneous and ephemeral nature of real-time 
interaction, the spoken form must be recognized and recalled very quickly. 
The written form must be accessed just as quickly while engaging in a 
reading or writing task, otherwise the plot might be lost. Therefore, fluency 
(or automaticity) plays a crucial role in spoken and written form recognition 
and production. 

Finally, Nation’s framework also includes knowledge of word parts, 
which refers to the fact that words may comprise affixes that are added to a 
root form (e.g. unbelievable, where the prefix un- and suffix -able are added 
to the root believe). Because affixes are likely to change not only the meaning 
of the root (e.g. un- means “not”) but also its word class (e.g. believe is a 
verb, but -able turns it into an adjective), proficient users need to be able to 
either break down or assemble the various parts of words so they can 
understand and use them. Knowledge of word parts has been identified as one 
of the most difficult word aspects, for both native and non-native speakers of 
English (Nation, 2013; Schmitt, 2010). 


Knowledge of Meaning 


The second main aspect of word knowledge is MEANING. As previously 
mentioned, L2 learners have usually gained substantial experience of the 
world prior to learning their L2, so their first task will be to learn the new 
forms attached to the meanings they already know. Receptive knowledge of 
the form—meaning link involves retrieving a meaning for a written or spoken 
L2 form, while productive knowledge entails producing the appropriate 
written or spoken L2 form for an intended meaning. In some cases, one form 
may encode multiple meanings (as is the case with polysemous words, such 
as bright and table) or various forms may have the same meaning, which L2 
speakers need to be aware of. 

Knowing a word’s concept and referents involves knowing the 
boundaries of the word meaning (in other words, what it includes and 
excludes). For example, the concept of the word ruminant is “any animal that 
brings back plant-based food from its stomach and chews it again” and some 
referents are Cows, goats, and sheep, but not cats and dogs. 

Word association knowledge may be defined as the ability to connect a 
word to other words, some of which could be used as substitutes. Those 
might be synonyms (words with the same meaning), homophones (words that 
sound the same), syntagms (words that occur together), paradigms (words 
that could be used as typical examples), etc. Research suggests that word 
association knowledge is indicative of the organization of the mental lexicon 
and the depth of vocabulary knowledge in general (e.g. Fitzpatrick & Munby, 
2014; Schmitt, 1998; Wolter, 2002). Meaning-based associations such as 


synonyms or paradigms characterize a well-developed mental lexicon typical 


of an advanced learner, whereas form-based associations such as 
homophones or syntagms characterize a sparser lexicon typical of a lower 


level of language proficiency (Cremer, Dingshoff, Beer, & Schoonen, 2010). 


Knowledge of Use 


The third main aspect of word knowledge is USE. It includes some of the 
most interesting components of word knowledge and is currently attracting 
the most attention in vocabulary research. In order to acquire vocabulary 
expertise, L2 learners must not only know the formal and semantic properties 
of words, but also, crucially, how those words behave in context when 
surrounded by other words. The analysis of large samples of real-world texts 
(corpora) has revealed that much of language is composed of patterns (e.g. 
Erman & Warren, 2000; Oppenheim, 2000), which might be of a 


grammatical, semantic, or formulaic nature. All word classes (verbs, adverbs, 


nouns, adjectives) follow grammatical patterns; for example, verbs might be 
transitive or intransitive. Knowledge of the grammatical pattern (or function) 
of a word is necessary to understand and use the word within a sentence. 
Words with similar meanings, for example, commonly co-occur (e.g. Ice- 
hockey is the most popular sport in Canada). In addition, many words also 
follow formulaic patterns and tend to occur with the same words in discourse, 
like collocations (e.g. spare time). The widespread occurrence of collocations 
in language is one of the most fascinating findings in the field of applied 
linguistics and will be explored in greater detail in the following section. 

In order to gain full knowledge of a word, L2 learners thus need to be 
able to recognize and use the words with which that word typically occurs in 
a sentence. Learners must also be aware of the constraints of use that might 
be associated with some words. This is known as knowledge of appropriacy 
of use, and refers to frequency and register issues. Words have different 


frequency distributions and the more frequent a word, the more likely it is to 


be understood and used in everyday speech. Having good intuitions of a 
word’s frequency will help a learner decide when to use it depending on the 
language proficiency level of the person they are addressing (e.g. a young 
child or older person). Gaining knowledge of a word’s register will involve, 
among other things, knowing whether it is more appropriate for formal or 
informal settings (e.g. an academic lecture or casual conversation), and for 
spoken or written discourse. Not all words in English have specific 
constraints of use. Many words (especially of the high-frequency variety) are 
neutral and can be used in a wide range of situations and discourse modes. 
Nevertheless, “constraints of use” is an essential component of word 
knowledge for those other words which may cause a great deal of confusion 
or embarrassment if used incorrectly, e.g. calling a man “skinny” instead of 
“slim.” 

Taken as a whole, vocabulary depth is a complex construct that is 
comprised of many components of knowledge. It is worth pointing out that 
those components are not mastered sequentially or in an “all or nothing” 
manner. Instead, they are more likely to be interrelated and develop in 
parallel, although it is not yet clear in what way and to what extent. Schmitt 
(2010, p. 38) provides a hypothetical illustration of how well the different 
knowledge components for a given word might be known at a given point in 


time, highlighting that the components are likely to develop at different rates 
(Figure 33.4). 
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Figure 33.4 Developing knowledge of a word (Schmitt, 2010, p. 38). 


Some components are thought to be relatively easy to acquire and 
suitable for explicit study (e.g. form and meaning), whereas others are more 
difficult and require repeated exposure to the words in context in order to be 
fully learned (e.g. collocations and constraints on use) (Schmitt, 2014). It is 
obviously unrealistic to expect highly proficient L2 speakers, or even native 
speakers, to know all words in English in such depth. Rather, it is more likely 
that only a relatively small proportion of the words they use are known in all 
the ways described above. However, acquiring as much knowledge of those 


aspects as possible is a desirable goal in order to gain vocabulary expertise. 


Formulaic Language 


What Is Formulaic Language? 


Traditionally, language has been seen as a system made of words that are 
arbitrary signs, and grammar rules that determine how these words can be 
combined to convey messages (Pinker, 1998). Based on this assumption, by 
combining syntactic rules with lexis, an infinite number of utterances can be 
created. While the creative nature of language remains unquestionable, more 
recent research suggests that native speakers store in memory a large number 
of ready-made formulaic sequences which they can rely on in order to 
produce and process language more efficiently (e.g. Nattinger & DeCarrico, 
1992; Wray, 2002). This means that even if we could produce a large number 
of novel utterances, we tend not to do so, and rely on the frequent use of 
formulaic sequences to a considerable extent. 

The term formulaic sequence has been defined as “a sequence < ... > of 
words or other elements which is, or appears to be, prefabricated: that is, 
stored and retrieved as a whole from memory at the time of use, rather than 
being subject to generation or analysis by the language grammar” (Wray, 
2002, p. 9). In other words, language users retain memory traces of 
frequently co-occurring words and are able to access these language chunks 
quickly when they need them. It seems that we rely on the formulaic 
sequences stored in the long-term memory to compensate for a limited 
working memory (i.e. the part of memory that we use to store and manipulate 
information for a short time), which makes formulaic language crucial for 
fluent language use (Conklin & Schmitt, 2012). 

There are different types of formulaic sequences that vary in length, 
structure, fixedness, and frequency of use. Some examples include proverbs 


(e.g. When in Rome, do as the Romans), idioms (e.g. kick the bucket), phrasal 


verbs (e.g. get up), collocations (e.g. dark night, meet expectations), and 
slogans (e.g. Just do it!) or songs. To capture all of these sequences, different 
identification criteria have been suggested. The most commonly used 
include: degree of non-compositionality (i.e. whether the meaning of the 
whole phrase can be derived from the sum of the meanings of its parts); 
storage in memory; single intonation curve when produced in speech; and 
conventionalization in society (usually shown by frequent occurrence in a 
corpus) (e.g. Erman & Warren, 2000; Schmitt, 2010; Wray, 2002). 


Why Is Formulaic Language Important? 


Pawley and Syder (1983) suggest that adult native speakers of English know 
at least several hundreds of thousands of formulaic sequences, which would 
mean that they know more word sequences than individual words. Both 
written and spoken discourse contain large numbers of formulaic sequences 
and some of these sequences are used very frequently. It is difficult to 
establish exact percentages because different researchers use different 
definitions of formulaic language, but most research suggests that formulaic 
language makes up between one-third and one-half of discourse (Conklin & 


Schmitt, 2012). Overall, the suggested percentages vary from about 20 


percent at the lower end (Sorhus, 1977) to up to 80 percent at the higher end 


(Oppenheim, 2000) and they vary between different types of formulaic 


language (Vilkaité, 2016). However, regardless of the percentages, there is 


pervasive evidence that formulaic sequences are very frequent in language. 
There is a general consensus that formulaic language is important for 

effective, appropriate, and fluent language usage (e.g. Barfield & Gyllstad, 

2009; Nattinger & DeCarrico, 1992; Wray, 2002). Schmitt (2010) 


summarizes some of these diverse communicative purposes: 


e expressing a message or an idea: Pull the plug on X = stop something, 


such as a project or a plan; 


e functional use: I (really) appreciate your 
thoughtfulness/kindness/doing X to express gratitude (Nattinger & 
DeCarrico, 1992); 


e social interaction (phatic communion): Did you really? and How 
interesting! to provide back channels and positive feedback to another 


speaker; 
e discourse organization: as a result of X to show cause and effect; 


e precise information transfer: Mayday, Mayday, Mayday as a technical 
phrase to indicate an emergency situation in aviation and maritime 


language. 


Because formulaic sequences are frequently and typically used for the above 
communicative purposes, they have become the default means of expression 
for these purposes within a speech community. This means that a learner’s 
interlocutors will not only accept conventionalized formulaic sequences in 
their discourse, but will also expect them as the norm and the most 
appropriate means of expression. It has also been widely attested that 


formulaic language aids and speeds up language processing for both speaker 


and hearer (Siyanova-Chanturia & Martinez, 2015). There is substantial 
evidence that formulaic sequences are processed faster in the brain and show 
clear predictability effects (as indicated by various reaction time experiments, 
and eye-tracking and brain imaging studies). These effects seem to be driven 
by the frequency of formulaic sequences in language, and their degree of 


familiarity for speakers (Siyanova-Chanturia, 2015). 


In terms of receptive skills (reading and listening), formulaic sequences 
are standard ways of realizing functions and expressing ideas. Therefore, 
knowledge of these sequences allows learners to process a piece of discourse 
more easily. This has been demonstrated particularly in reading with eye- 
tracking studies, which show that formulaic sequences are read faster than 


novel phrases (e.g. Siyanova-Chanturia, Conklin, & Schmitt, 2011; Sonbul, 


2015). In relation to language production (especially speaking), formulaic 
language facilitates the production of fluent speech despite the cognitive 
constraints of operating online. For example, Kuiper (2000) and Kuiper and 
Haggo (1984) found that the discourse of people who need to produce fluent 
speech under severe time pressure, such as auctioneers, was largely formulaic 
in nature. Wray (2002) also claimed that in certain situations where language 
has to be produced very quickly (such as in sports commentaries or 
simultaneous translation), speakers tend to rely heavily on formulaic 


sequences in order to overcome the huge load on their working memory. 


Use of Formulaic Language by Second Language Learners 


When analyzing formulaic language produced by non-native speakers, two 
different aspects need to be considered: amount of use and appropriacy of 
use. As far as the amount of use is concerned, it is not the case that L2 
learners do not use formulaic sequences at all. Researchers looking at mostly 
written production of second language learners have shown that learners 
(depending on the sequences) underuse, overuse, or misuse them when 


compared to native speakers (Paquot & Granger, 2012). They also tend to use 


large numbers of collocations that are easy for them (Paquot & Granger, 
2012), and avoid certain formulaic sequences that they find difficult to use. 
For example, Dagut and Laufer’s (1985) study of the use of phrasal verbs 
demonstrated that L1 Hebrew learners of L2 English avoided using phrasal 
verbs in their writing when using English, as these did not exist in their first 
language. These studies suggest that failure to use formulaic sequences in a 
native-like way makes learners’ language sound unnatural. 

However, it is not only the overall number of formulaic sequences that is 
important. Appropriate use of expected sequences is equally crucial, but is 
not an easy task to achieve. Analysis of learner language indicates that about 
one-third of the collocations produced by language learners are not judged 
acceptable by native speakers. This finding holds regardless of the learners’ 
level of proficiency, with even advanced learners making mistakes in their 


use of collocations (Paquot & Granger, 2012). This is problematic, as it was 


shown that malformed collocations (such as cheap price instead of low price) 


slows down the processing for the native speakers (Millar, 2011), and thus 


can hinder communication. 


A very common reason for using unacceptable word sequences is the 
influence of the learner’s L1. If formulaic sequences are congruent in the L1 
and L2, they are easier to produce and process, but if they are different in the 
two languages, the L1 tends to interfere and slow down processing (see 
Bahns & Eldaw, 1993 for producing and Wolter & Gyllstad, 2011 for 


processing). 


How Can L2 Learners Improve Their Use of Formulaic Sequences? 


Because native speakers use a large number of formulaic sequences, L2 
learners are faced with the enormous task of determining which word 
combinations sound native-like. It is not surprising that knowledge of 
formulaic sequences lags behind the knowledge of single words (Bahns & 
Eldaw, 1993). Why is acquiring formulaic language in the L2 so hard, when 
it is such an effortless process in the L1? Wray (2002) notes that, when 
starting to learn an L2, mature learners have different cognitive abilities and 
are usually literate. This pushes them to focus on word-level units and 
process language more analytically. Also, their learning situation is typically 
very different in that the L2 is usually learned in a classroom with much less 
exposure to naturally occurring language and much more social pressure. 
Regardless of that, L2 learners can and do acquire formulaic sequences. 
Several ways of learning and teaching formulaic sequences have been 
suggested by previous research, with differing degrees of success. Research 
shows that prolonged stays in a country where the L2 is spoken, as well as 
the motivation to be integrated into the L2 speech community helps to 
increase the use of appropriate formulaic sequences (Adolphs & Durrow, 
2004; Déornyei, Durrow, & Zahran, 2004; Macis & Schmitt, 2017). 
Nevertheless, the vast majority of language learners across the world do not 
have the opportunity to live in a country where the L2 is spoken. Fortunately, 
exposure can be achieved easily from home via the use of technology. For 
example, the language used on television seems to correspond to everyday 


spoken English in terms of the formulaic sequences it contains (Lin, 2014). 


Other more traditional methods of teaching and learning formulaic 
sequences also seem to be effective. Some studies suggest that the same 
teaching methods can be applied to both single words and formulaic 
sequences (Alali & Schmitt, 2012; Webb & Kagimoto, 2009). However, even 


if vocabulary exercises are effective for learning formulaic sequences, the 


problem of choosing which sequences to teach remains. In order to solve this 
problem, researchers have created lists of the most frequent, and therefore 
most important, items to learn. Examples include the PHaVE list for phrasal 
verbs (Garnier & Schmitt, 2015) and the PHRASE list for non-transparent 
formulaic sequences (Martinez & Schmitt, 2012). These lists are a valuable 
Starting point to give learners and teachers some guidance. Nevertheless, 
learning phrases beyond these lists remains a challenge, especially as many 
other types of formulaic sequence have no available lists. 

It seems that a large number of formulaic sequences have to be learned 
from exposure when reading or listening to language. Research on incidental 
acquisition (i.e. learning without intention) of formulaic language shows that, 
although the rate of acquisition is slow, learners do acquire formulaic 
sequences from reading even without consciously focusing on learning these 
items (Pellicer-Sanchez, 2017; Webb, Newton, & Chang, 2013). So, 
maximizing exposure seems to be a crucial learning principle. Overall, it 
appears that formulaic language takes a long time to acquire, but it is also a 
hallmark of the highest stages of language mastery. It is indeed the 
appropriate use of formulaic language that distinguishes an expert language 


user from a novice or intermediate one. 


Practical Suggestions for Gaining L2 
Expertise 


The above discussion has emphasized that learning enough vocabulary is a 
big challenge for learners. Explicit study of individual words and formulaic 
sequences is the quickest and most effective way of learning this vocabulary. 
But this will only ever cover a limited number of the lexical items necessary 
to use a second language. It will also be necessary to maximize the learner’s 
exposure to English in a number of ways: (a) reading extensively; (b) 
watching TV and movies; (c) using social media; (d) interacting with native 
speakers of English; and (e) spending time in an English-speaking country. 
Because spending time in an English-speaking country is impractical for 
most students, and access to native speakers of English may be severely 
limited, reading, watching TV and movies, and using social media may be 
seen as the most convenient ways to increase a learner’s exposure to English. 

For beginners, the use of graded readers (simplified books that are 
specially created for second language learners) is a sensible starting point, as 
these have the advantage of providing substantial language input at an early 
stage of a learner’s development. They can also help to establish a desirable 
long-term reading habit in the L2. As learners’ proficiency increases, they 
will naturally want to move on to authentic texts. However, the jump from 
graded readers to authentic texts can be a daunting one and narrow reading 
might help ease this transition. Narrow reading refers to reading numerous 
texts on the same topic, with the advantage that much vocabulary is recycled, 


making the topic increasingly easier to process. Narrow reading can be 


achieved by following a continuing story in a newspaper or by reading 
magazines focusing on a particular theme. After this stage, the most effective 
way to increase language input is through extensive reading, which simply 
refers to reading large quantities of text from diverse sources and of various 
topics. By reading extensively, the chances of learning new words and 
consolidating knowledge of partially known words increase substantially. 
Watching TV and movies provides not only entertainment, but also the 
opportunity for learners to develop their language skills and increase their 


vocabulary sizes (King, 2002). Through films, learners can see how people 


communicate in real life in different conversational contexts, and get a feel 
for the kind of vocabulary they use in each context. Using subtitles is a useful 


source of vocabulary learning (d’ Ydewalle & Pavakanun, 1995). In this way, 


learners can match the written subtitles to the words spoken on screen, which 
enables them to further develop their vocabulary. Keeping a notebook, and 
writing down newly encountered words or phrases and looking up their 
meaning later are also very useful activities for learners of all levels (Walters 
& Bozkurt, 2009). 

Research also suggests that using social media (Facebook, Twitter, and 
MySpace) in the L2 has been found to facilitate knowledge of collocations 
and phrasal verbs (Garnier & Schmitt, 2016; Gonzalez-Fernandez & Schmitt, 
2015). In addition, applications like Vocabulary.com, Phrasal Verbs 
Machine, and English Vocabulary in Use are an excellent resource for 
improving learners’ vocabulary, and are immediately accessible. They reflect 
the type of exposure which makes language more engaging and personal, and 
thus perhaps even more conducive to learning. 

All of these input-maximizing approaches will not only promote the 


development of a larger vocabulary, but will also provide the contexts which 


facilitate depth of knowledge of the various lexical items. 


Conclusion 


This chapter has highlighted what we believe are the three key requirements 
in order to reach expert knowledge of L2 vocabulary. A large vocabulary size 
is necessary to be able to function well in English (Nation, 2006), but learners 
also need to know words well, and how they combine in discourse, in order 
to use them productively, appropriately, and fluently. 

The discussion presented here also calls for more research in the area. 
Nation’s (2006) paper is still held as the standard reference for lexical 
requirements in English. However, ten years on, many of his assumptions are 
now being questioned, especially whether his reported figures are applicable 
to all kinds of reading and listening contexts and materials. Furthermore, the 
studies on lexical coverage reported above have not taken into account the 
variability among individual learners and their language skills. Consequently, 
we propose that future research should revisit Nation’s figures with the aim 
of updating estimates of how much vocabulary is needed to read and listen to 
a variety of discourse types, and providing more finely tuned percentages. 

The field also lacks reliable and comprehensive measures of vocabulary 
depth. Only a few studies have examined the acquisition of multiple word 


knowledge components concurrently (e.g. Schmitt, 1998; Webb, 2007), so it 


is still unclear how the different components of vocabulary knowledge are 
acquired and relate to each other. Gonzdalez-Fernandez (in preparation) 
explored the order of acquisition of four word knowledge components (the 
form—meaning link, derivatives, polysemy, and collocations) in Spanish 


learners of English. Results revealed an unexpected order of difficulty for 


some aspects, with derivative knowledge (i.e. “word parts” in Table 33.2) 
proving to be particularly difficult. They also showed that the vocabulary 
knowledge components are all interrelated, but at different levels of strength. 
However, these are only initial results, applicable to one specific group of 
learners, thus indicating the need for more research in order to arrive at a 
more detailed and accurate description of vocabulary acquisition. 

Finally, it is still unclear what the most effective way to teach formulaic 
language is. Although some researchers have already addressed this issue to a 
certain extent (Alali & Schmitt, 2012; Webb & Kagimoto, 2009), these and 


similar studies suffer from a number of limitations. Some of them have used 


a small number of participants, whereas others have examined only one 
particular type of formulaic language (e.g. collocations). Only further 
research with larger populations, a wider range of teaching methods, and a 
more diverse selection of formulaic sequences can help us identify the 


principles of truly effective teaching techniques. 
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Introduction 


The scientific study of expertise in sport has grown markedly in popularity 
over recent decades. This growth has occurred partly in line with the 
expansion of sports science as an academic field, partly as a result of the 
increasing amounts of funding involved in high-performance sport, and partly 
due to growing public interest in gaining a better understanding of what 
makes athletes experts. In this chapter, we present an overview of research 
that has been undertaken on expertise in sport over recent decades. Our aim is 
to provide a summary of the field for the broader readership, as well as to 
highlight some of the key challenges in sport expertise research and suggest 
avenues for further academic endeavor. In light of the vast amounts of 
empirical work undertaken on expertise in sport from different disciplinary 
perspectives, we delimit the scope of our overview. Our discussions will 
mostly focus on some of the psychological processes that underpin expert 
performance in sport, with a particular focus on the perceptual-cognitive- 
motor skills that underpin anticipation and decision making. 

The chapter is presented in three parts. First, we review research that has 
examined the developmental history profiles of elite athletes, although we 
should acknowledge that not all data that contribute to this debate are 
gathered from truly elite athletes. We summarize research that has tested 
some of the central claims of the theory of deliberate practice proposed by 
Ericsson, Krampe, and Tesch-R6émer (1993) and report key observations on 
the practice patterns of elite athletes across several sports. The importance of 


engagement in a high quantity and quality of practice and sport-specific play 


is highlighted as a precursor to the development of expertise and we discuss 
how specific types of early sport activities relate to the development of 
perceptual-cognitive skills. Second, we review some of the psychological 
adaptations that occur as a result of prolonged engagement in various types of 
practice activities. We show how athletes adapt to the unique constraints of 
their sport with these adaptations reflecting the demands of their performance 
environment. In this second section, we focus on the important perceptual- 
cognitive skills that underpin anticipation and decision making in dynamic, 
time-constrained sports. Finally, we consider the implications of empirical 
research on expertise in sport for policy development and investment in 
systems and structures in high-performance sport. The implications of this 


research are considered both for talent search and for talent development. 


What Type of Practice Experiences Describe 
the Developmental Pathways to Performance 
Excellence in Sport? 


Practice History Profiles (the Macro Level) 


Those who reach the pinnacle of sporting achievement are often believed to 
have been born with “gifts from the gods” and with the good fortune to have 
been exposed to their target sport at a very young age. It is not our intention 
to engage in the perennial nature vs. nurture debate (for an extended 
discussion, see Ford, Hodges, & Williams, 2013), but it is evident that 
anecdotal testimony, mainly in the form of biographical information, suggests 
that elite athletes invest significant time and effort into their sport on the path 
to excellence. This latter observation suggests that high-achieving athletes 
develop skills through a complex interaction between genes and opportunities 
to express those genes through exposure to the right environment and practice 
opportunities. Although the idea that “practice with zeal” leads to 
performance improvement had been around for almost a century (Thorndike, 
1912), it was not until Ericsson and colleagues published their seminal paper 
on deliberate practice in 1993 that a conceptual framework and a 
methodology emerged that enabled the importance of practice to be examined 
empirically. 

Deliberate practice was defined as specific, purposeful, and challenging 
practice of an aspect of performance that requires improvement (Ericsson et 
al., 1993). It requires explicit goals, effortful repetition of the activity in 
question, supported by feedback. It was also claimed that deliberate practice 
was not inherently enjoyable and did not offer any immediate reward 
(Ericsson, 2003, 2007, 2008). In their initial study, Ericsson and colleagues 
used interviews and diary methods to examine retrospectively the 
developmental history profiles of performers at an elite music academy in 


Berlin. Violinists were categorized into four groups based on their level of 


attainment, namely, the best violinists in the academy, good violinists, music 
teachers, and professionals who performed in a world-class orchestra. The 
mean start age for practicing the violin across groups was ~8 years. However, 
across development, participants in the four groups had accumulated 
significantly different practice hours. The best violinists in the academy and 
the world-class orchestra violinists had accumulated 7,410 and 7,366 hours in 
practice activity respectively by the age of 18 years. In contrast, by the same 
age, the good violinists and the music teachers had accumulated only 5,301 
and 5,420 hours respectively. In a follow-up study, expert pianists were 
reported to have started in the domain at ~6 years of age and accumulated 
7,606 hours by 18 years of age, which was significantly more than the 1,606 
hours accumulated by amateur pianists. The notion that long and intensive 
periods of practice are a prerequisite to the development of expertise has 
subsequently been reported across numerous other domains (for detailed 


reviews, see Ericsson, Charness, Hoffman, & Feltovich, 2006; Starkes & 


Ericsson, 2003), yet the amount of variance in performance that may be 
meaningfully accounted for by this factor remains somewhat contentious 
(Hambrick & Tucker-Drob, 2015; Hambrick et al., 2014; Macnamara, 
Moreau, & Hambrick, 2016). 

Since the seminal work of Ericsson and colleagues, several researchers 
have employed questionnaires and training diaries to retrospectively study the 
developmental history profiles of elite athletes (for recent reviews, see Baker 
& Young, 2014; Coté, Murphy-Mills, & Abernethy, 2012; Ford, Coughlan, 
Hodges, & Williams, 2015; Ford et al., 2013). The main focus has been on 


identifying the nature and quantity of practice activities in which athletes 


have engaged across different stages of development. Scientists have 


differentiated between activities in the main sport of eventual expertise, 


which includes coach-led or team practice, non-coach-led practice or play, 
which may be engaged in a group or alone, and competition activity, as well 
as activity in other sports (Ford & Williams, 2012, 2013; Ford et al., 2012). 

Starkes and colleagues were the first to explore the applicability of 
deliberate practice theory in the domain of sport (e.g. Starkes, Deakin, Allard, 
Hodges, & Hayes, 1996). In one study, they used similar retrospective recall 
methods to those used by Ericsson and colleagues to examine the practice 
history profiles of elite (international level) and sub-elite (club level) 
wrestlers (Hodges & Starkes, 1996). The wrestlers began participating in 
their sport relatively late when compared to musicians (~13 years); however, 
similar to the musicians, after 12 years in their sport, the elite wrestlers had 
accumulated 5,882 hours of practice (averaging 38 hours per week) compared 
to sub-elite wrestlers who had accumulated 3,571 hours (averaging 20 hours 
per week). Helsen and colleagues (Helsen, Starkes, & Hodges, 1998) 
extended this work to the sport of soccer (and field hockey) by comparing the 
practice history profiles of international, national, and provincial level soccer 
players in Belgium. The soccer players began participating in the sport at ~5 
years of age and by the age of 18 years the three groups of players had 
accumulated approximately 6,200, 5,000, and 3,900 hours respectively (for 
detailed and historical reviews of research in sport, see Baker & Young, 
2014; Ford et al., 2015; Ward, Hodges, Starkes, & Williams, 2007; Ward, 
Hodges, Williams, & Starkes, 2004; Williams & Ford, 2008). 


Researchers in sport have identified two issues with the original 


definition of deliberate practice. The first is with respect to the claim that 
deliberate practice is not inherently enjoyable, and the second concerns the 
issue as to whether it should be conducted alone (or one-to-one with a coach, 


or in a group or team). Both of these issues remain largely unresolved. In 


sports, athletes have rated some sport practice activities (e.g. tactical work, 
sparring) high for enjoyment and relevance to improving performance (e.g. 


Helsen et al., 1998; Hodges & Starkes, 1996). In contrast, these same athletes 


rated fitness activities (e.g. weights, running) low for enjoyment and high for 
relevance, in line with the theory. In a diary study of elite triathletes that was 
designed to overcome potential biases with aggregated retrospective affective 
ratings of activities, those physical activities that were rated as highly 
relevant to improving performance on a day-to-day basis showed no 
correlation to ratings of enjoyment, despite associations with effort and 
satisfaction (Hodges, Kerr, Starkes, & Nananidou, 2004). There have been a 
number of attempts to explain these sport-related differences with respect to 
enjoyment. It has been argued that some sport-specific practice activities do 
not contain all of the characteristics of deliberate practice, such as having 
explicit goals designed to improve performance, causing overall ratings of 
enjoyment to become inflated (Ericsson, 2016; Ford et al., 2013, 2015), 
Because sport is an inherently social activity and engaging in it leads to 
positive outcomes, it has been suggested that athletes may rate the social 
interaction during the activity or the consequences of the activity as 
enjoyable, instead of the actual activity (Ericsson, 1996). When the 
consequences of the activity have been differentiated from its inherent 
enjoyment, ratings of enjoyment are generally lower, yet still relatively high 
(Hodges et al., 2004; Ward et al., 2007). Moreover, individual activities and 
sports (such as those involved in figure skating, swimming, or triathlons) in 
which there should be less social interaction compared to team sports, are still 


rated high for enjoyment (Hodges et al., 2004; Starkes et al., 1996). In view 


of these issues, inherent enjoyment (positive or otherwise) is probably viewed 


as the least important feature of “deliberate practice,’ in comparison to 
features such as effort and relevance to performance improvement. 

The second issue concerns the importance of individual practice in the 
context of sport where team/group or coach-led practices typically dominate 
practice time. Helsen et al. (1998) showed that amount of time in individual 
practice mattered in the early childhood years with respect to differentiating 
across professional and amateur players in soccer in Belgium, but that after 
the age of 12 years, it was team practice that was the best discriminator. 
Team practice better discriminated skilled from less skilled athletes, in 
comparison to individual practice hours, in other studies of sport skill, 


including wrestling (Hodges & Starkes, 1996) and youth-level soccer (Ward 


et al., 2007). Much of the current debate concerning relevant sport-related 
practice activities centers on the importance of self-designed vs. coach-led 
practice (Ericsson, 2016; Macnamara et al., 2016), which often co-varies with 
individual vs. team practice, but also relates to differences between what has 
been termed “play” and “practice” (e.g. Hendry, Crocker, & Hodges, 2014). 
We return to this discussion below. 

In recent research, scientists in the domain of sport have attempted to 
elaborate on the original deliberate practice framework (Ericsson et al., 
1993).There have been attempts to identify deliberate practice features at the 
micro- rather than the macro-level (e.g. Ford, Yates, & Williams, 2010). 
There have also been efforts to identify the types of practice activities that 
lead to the development of specific skills, such as anticipation and decision 
making (e.g. Low, Williams, McRobert, & Ford, 2013). 


Deliberate Practice (the Micro-Level) 


There have been several attempts to explore the micro-structure of deliberate 


practice in sport, including practice periodization and how athletes schedule 


periods of rest from physical work (e.g. Baker, Cote, & Abernethy, 2003a; 
Ford et al., 2010). One approach that has recently been adopted is to study the 


behaviors of athletes when practicing specific skills, to determine whether 
their choice of practice activities adheres to the principles of deliberate 


practice (Coughlan, Williams, McRobert, & Ford, 2013). Groups of expert 


and intermediate level Gaelic footballers were asked to practice two types of 
kicks toward a goal from the hand and off the ground using their stronger and 
weaker foot. A traditional experimental design was employed where the 
athletes were tested on an initial pre-test, during acquisition, and on post- and 
retention tests to assess learning. Players were free to practice the skills in 
whatever manner they deemed most appropriate across four practice sessions. 
Ratings of mental effort and enjoyment were taken during the sessions, as 
well as outcome scores. The expert group predominantly practiced the skill 
they were weakest at and improved performance across the test sessions more 
so than intermediate players, who in contrast spent the majority of time 
practicing their strongest skill (see also Hodges, Edwards, Luttin, & 


Bowcock, 2011). The expert participants also rated their practice as being 


more effortful and less enjoyable than those in the intermediate groups, in 
line with deliberate practice theory. 

Other researchers have assessed how elite athletes engage in practice 
and to test whether their practice activities, on a micro-level, follow the tenets 


of deliberate practice. In one study, expert and intermediate players engaged 


in practice of a Gaelic football kicking skill rated as most relevant to 
improvement of overall performance (Coughlan, 2016). Both groups selected 
the kick from the hands with the non-dominant foot. They practiced the skill 
across three sessions in a week and skill acquisition was assessed in a delayed 
retention test. As illustrated in Figure 34.1, only the expert group improved 
kicking performance between the pre- and retention test, in comparison to a 
control group of experts that did not practice and the intermediate group. The 
expert group rated their practice higher for mental effort and lower for 
enjoyment compared to the intermediate group. The expert group also 
engaged in greater amounts of cognitive processing through reflection on 
their performance before, during, after, and between practice sessions when 


compared to the intermediate group. 


@ Pre-test 
® Retention test 


Points scored 


Expert Intermediate Expert-control 
Group 


Figure 34.1 Mean (SD) accuracy scores for kicks with the non-dominant 
foot by the expert, intermediate, and expert-control groups in the pre- and 
retention test. 


In a further study, two groups of youth, intermediate-level performers 
matched for age, playing experience, and scores on the Self-Regulation of 


Learning Self-Report Scale (SRL-SRS; Toering, Elferink-Gemser, Jonker, 


Van Heuvelen, & Visscher, 2012) practiced a kicking skill they had selected 
as most relevant to improving their overall current performance (Coughlan, 
2016). A training group received an intervention designed to increase the 
extent to which they engaged in thoughtful analysis of their own performance 
before, during, and after kicks and practice, whereas the control group 
engaged in practice without this intervention. Both groups significantly 
improved their ability to kick the ball at goal from pre- to post-test and 
maintained their improved level of performance in a retention test. However, 
the training group’s kicking accuracy was significantly better than that of the 
control group in both the post- and retention test. Moreover, the effect of the 
intervention was apparent during practice, with the training group 
demonstrating greater improvement across each session compared to the 
control group. The training group also reported increased mental effort during 
practice compared to the control group, but the groups were not differentiated 
for enjoyment. Compared to the pre-test, the training group engaged in more 
evaluation of their performance on the retention test, whereas the control 
group did not change their thought processes from pre- to retention test. A 


summary of these findings is presented in Figure 34.2. 
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Figure 34.2 Mean (SD) accuracy scores for kicks with the non-dominant 


foot by the intervention and control groups in the pre- and retention test. 


We argue that this move away from a reliance on descriptive, 
retrospective, correlational based work on deliberate practice is a positive 
development in this field. The approach presents an exciting opportunity to 
test some of the key predictions of deliberate practice theory. These practice 
manipulations and quasi-experimental designs will lead to information about 
best quality practice that would have immediate relevance for coaches. 
Concurrently, these micro-structure approaches should help steer researchers 
(and practitioners) away from focusing on amount of practice and counting 


(and accruing) practice hours. 


Relationships between Practice Activities and Specific Perceptual- 

Cognitive Skills 
Another recent development in the uncovering of practice profiles that define 
expert athletes, has been the study of activities that lead to the development 
of specific perceptual-cognitive skills rather than performance more broadly. 
One approach to address this issue has been to use standard film-based tests 
of anticipation to classify athletes into high and low performing groups based 
on their performance on tests of perceptual-cognitive skill. After the within- 
task criterion is used to stratify participants, the practice history profiles of 
these athletes are compared across the groups to identify what types of 
activities might have facilitated development of this skill (e.g. Low et al., 
2013). 

Using this stratification approach, based on performance on the 
anticipation test, elite soccer players aged 18 years were subdivided into 
skilled “high performing” and “low performing” groups and compared to a 
control group of recreational soccer players (Williams, Ward, Bell-Walker, & 


Ford, 2012b). No differences were observed between the three groups in the 


amount of hours accumulated per year in practice activities or in competition. 
However, the “high performing” group had accumulated significantly more 
hours in non-coach-led play related activity in soccer compared with their 
“low performing” elite counterparts over their last six years of engagement in 
the sport. Both elite groups had accumulated significantly more hours in this 
coach-led play compared to the non-elite group. Similar findings were 
reported by Roca, Williams, and Ford (2012) using samples of soccer players 


stratified into high and low performing groups based on their performance on 


a film-based test of anticipation and decision making. During childhood 
(6-12 years), the high performing group averaged more hours in soccer- 
specific play compared to both the low performing group and recreational 
players. In adolescence (13-18 years), both skilled groups accumulated more 
hours in soccer-specific practice and competition when compared with the 
recreational players. By 18 years, the high performing group had 
accumulated more hours in soccer activity compared to the low performing 
group, with both groups accumulating more hours compared to the 
recreational players. A linear regression analysis showed that ~22 percent of 
the variance in performance on the test of anticipation was accounted for by 
the hours accumulated in non-coach-led, soccer-specific play during 
childhood, with a further 13 percent of the variance being due to the hours 
accumulated in coach-led, soccer-specific practice during adolescence. 

These data suggest that the hours accumulated in non-coach-led, sport- 
specific play had a positive impact on the development of perceptual- 
cognitive skills. These types of activities, such as street or backyard soccer, 
may provide opportunities for discovery learning where technical and tactical 
skills are practiced together in a dynamic and integrated manner, similar to 
the competitive match situation. Children may recreate the types of activities 
seen during match play, creating opportunities to mimic the demands of 
competition. Moreover, non-coach-led practice activities are likely to be 
devoid of explicit and augmented instruction and feedback, although not 
necessarily devoid of thoughtful analysis and reflection. In contrast, there is 
evidence to suggest that coach-led practice activities tend to overemphasize 
drill and grid-based activities rather than activities that more naturally mimic 
the demands experienced during competition, such as small-sided and 


conditioned games (Ford et al., 2010). However, it is worth considering that 


these skills are a result of the additional engagement in sport-specific 
activities, over and above what is typically undertaken in a team setting. We 
know that in the UK, at least, practice hours in an Academy setting are quite 
fixed and, as such, it is difficult to differentiate players based on time in team 
practice because there is little to no variation in this measure (Hendry et al., 
2014). It will be important in future research to control for overall 
engagement hours in the sport, perhaps determining proportion of time in 
various activities when overall engagement hours are matched across 
individuals. 

There are also advantages to studying how performance on measures of 
anticipation and tactical skill co-vary over time with changes in practice and 
play, such that specific adaptations can be assessed in concert with specific 
changes in training habits and behaviors. In one study of elite youth soccer 
players in Scotland, attempts were made to relate practice and/or play-type 
activities to coach ratings of various skills (i.e. tactical, physical, technical, 
and creative; Hendry & Hodges, in preparation). No significant relationships 
were reported between these skills and practice and play activities reported in 
practice history profiles. Only among a small group (n = 32) who were later 
retained on professional contracts at age 16 years were relationships evident 
between total accumulated hours in soccer practice and tactical (r = 0.47, p 
<0.05) and technical (r = 0.59, p <0.01) skill ratings. Although there are some 
issues with subjective measures of skill and attrition when elite athletes are 
tracked over time, this type of research has the potential to better gauge 
relationships between sports activities and the development of specific skills. 

Although evidence supporting the importance of engagement in sport- 
specific training activities is compelling (for a review, see Ward et al., 2004), 


and strongly grounded in theories of specificity of practice (Lee, 1988; 


Proteau, 1992), some correlational evidence suggests that engagement in 
practice activities outside of the target domain (i.e. other sports and physical 
activities) may positively impact on the development of perceptual-cognitive 
skills. Published reports suggest that athletes at elite training institutes in 
Australia who are perceived by coaches to be good at anticipation in 
adulthood have typically accumulated considerable experience in other 
related sports during their earlier development (Weissensteiner, Abernethy, & 
Farrow, 2009; Weissensteiner, Abernethy, Farrow, & Miller, 2008). 
Similarly, there has been evidence showing that the best decision makers in 
sports, again based on an Australian sample of internationally renowned team 
athletes, were those who had early sampling/diversified experiences in other 
sports before entering their main sport and that practice hours in the main 
sport could be compensated for by hours in other, potentially related sports 
(Baker, Cote, & Abernethy, 2003b). Of course, it is possible that those 
individuals who have acquired good perceptual-cognitive skills in one 
domain are more likely to find ways to exploit those skills in other domains 
too. 

The authors of these studies on sporting diversity speculate that there 
may be some transfer of perceptual-cognitive skills across related sports. 
There are data to suggest that there is transfer of perceptual-cognitive skill 
from one related sport to another, such as field hockey to soccer (Smeeton, 
Ward, & Williams, 2004) or from baseball to cricket (Moore & Miiller, 
2014), yet these findings should not be taken to imply that engagement in 
related sports has advantages over participation in the targeted sport for 
excellence. There is evidence to suggest that the development of perceptual- 
cognitive skill is specific both to a particular position in a team (e.g. 


Williams, Ward, Smeeton, & Ward, 2008) and to one’s particular role within 


a sport (Catteeuw, Helsen, Gilis, & Wagemans, 2009). Clearly, further 
research is needed to improve our understanding of the specificity and 
generalizability of perceptual-cognitive skills in sport and across other 


domains. 


Plasticity and Adaptability: Why Do Athletes 
Look Like They Have “All the Time in the 
World”? 


In fast ball sports such as basketball and tennis, athletes have to contend with 
severe and stringent temporal constraints. In tennis, for example, an 
opponent’s serve often exceeds 130 mph leading to the situation where 
players are unable to react to stimuli as they evolve in the display. Players 
instead must try and predict the future by attempting to anticipate what the 
opponent will do next based on information available prior to the event itself. 
An important question that has been addressed by scientists is how do elite 
athletes learn to cope with these severe time pressures and perform as if they 
have “all the time in the world”? In other words, what types of adaptations 
need to occur in order that athletes are able to cope with the demands placed 
upon them in high-performance sport? The key finding is that engagement in 
vast amounts of practice and play-related activities in sport leads to specific 
adaptations in memory that are unique to that performance environment 


(Ericsson & Williams, 2007). In high-performance sport, exceptional levels 


of plasticity and adaptability are prerequisites to progression to higher levels. 
In this section, we highlight some of the key perceptual-cognitive skills that 


underpin anticipation in sport. 


Visual-Information Pick-Up: Perceptual-Cognitive Expertise 


Numerous published reports illustrate that expert athletes are better than their 
less expert counterparts at using vision to pick up relevant information to 
guide perception and action (Causer, Janelle, Vickers, & Williams, 2012; 


Williams, Davids, & Williams, 1999). Scientists have used portable, head- 


mounted eye movement registration systems to record visual point of gaze 
during performance on various sports tasks, both in the laboratory, often 
using film-based simulations, and in the field (Williams, Janelle, & Davids, 
2004). Typically, when compared to less expert performers, expert athletes 
fixate on areas of the display that are more informative, using more efficient 
and effective search strategies. In some instances, experts employ more 
fixations of shorter duration and in others they employ fewer fixations of 
longer duration; the most effective search pattern seems to be dependent on 
the sport and situation (e.g. Roca, Ford, McRobert, & Williams, 2013), 
number of athletes/space involved (e.g. Vaeyens, Lenoir, Williams, Mazyn, 
& Philippaerts, 2007), and athlete role (e.g. see Helsen & Starkes, 1999 vs. 


Williams & Davids, 1998) in team sports, as well as stressors such as anxiety 


(e.g. Vater, Roca, & Williams, 2016) and fatigue (Casanaova et al., 2013). 


Expert—novice differences in gaze behaviors have not always been reported, 
with in these instances the superior performance demonstrated by experts 
assumed to be a product of more effective information pick-up from each 
fixation or better use of so-called “visual pivots” where attention is 
distributed outside of the visual fovea (Ripoll, Kerlirzin, Steine, & Reine, 
1995; Vaeyens et al., 2007; Williams & Davids, 1998). 


The superior gaze behavior of expert athletes is assumed to facilitate the 
more effective pick-up of within- and between-person relative motion 
information, which facilitates pattern recognition. The pick-up of information 
from a single opponent, classically referred to as postural cue usage, has been 
demonstrated in numerous sports, with findings highlighting the perceiver’s 
greater attunement to low-level biological motion information (Abernethy, 
Gill, Parks, & Packer, 2001; Huys, Smeeton, Hodges, Beek, & Williams, 
2008; Ward, Williams, & Bennett, 2002). The film-based temporal occlusion 
approach has been the most frequently used method in this area. An opponent 
is normally filmed performing an action (e.g. the serve in tennis) from the 
athlete’s viewing perspective and the action is then played back to the 
observer on a large video screen. The action is then occluded at various time 
points relative to ball—racket or ball—foot contact (e.g. before, at, after) and 
the observer is required to anticipate where the opponent will place the ball. 
The classic findings are that experts demonstrate superior anticipation, 
particularly in the early (pre-contact) occlusion conditions, with performance 
levels being significantly above chance. The conclusions are that experts are 
better able to pick up early occurring (i.e. advance) information from the 
postural orientation of an opponent, with this skill assumed to be fundamental 
to anticipation. This skill was originally highlighted in tennis (Jones & Miles, 
1978) with similar conclusions subsequently reported in other sports, 
including cricket (McRobert, Williams, Ward, & Eccles, 2009), squash 
(Abernethy & Russell, 1987), badminton (Abernethy & Zawi, 2007), and 
baseball (Moore & Miiller, 2014). 

The pick-up of between-person, or inter-individual relative motion stems 
from the original work on familiarity detection using the classic recall and 


recognition paradigms (Chase & Simon, 1973). In the domain of sport, the 


typical approach has been to show athletes video footage of evolving 
sequences of play from the perspective of a central defensive player (in sports 
such as field hockey, soccer, handball, and football) and then to ask players 
either to recall players’ positions at the end of the sequence or to make a 
familiarity-based judgment as to whether the sequence observed was 
presented in an earlier viewing phase. The assumption, albeit not fully 
substantiated, is that if an athlete can identify familiarity early in the 
sequence of play, it will enable a more accurate prediction of the eventual 
outcome, thereby facilitating anticipation. Skilled athletes have been reported 


to be more accurate both in recalling player positions (Williams, Davids, 


Burwitz, & Williams, 1994) and in recognizing previously viewed sequences 


of play (e.g. Gorman, Abernethy, & Farrow, 2012; North, Ward, Ericsson, & 
Williams, 2011; Williams, North, & Hope, 2012a). Skilled athletes are 


assumed to pick up relational information between the positions and 


movements of opponents in order to identify “patterns” or familiarity in 
evolving sequences of play. The proposal is that skilled athletes are able to 
pick up low-level, inter-individual relational information between players and 
then to match this information with some high-order template in memory 


based on a pattern-matching process (Dittrich, 1999). 


It’s All About the Context: High-Level Cognitive Processes 


In addition to their superior ability to pick up information from the visual 
scene, expert athletes are more accurate than less expert counterparts in their 
expectations of what will likely happen next and are better able to integrate 
higher-order cognitive information relating to the specific context of the 
situation at hand. Alain and colleagues published seminal work on the use of 
situational/event probabilities in sport. In a series of studies, these authors 
manipulated the probabilities of certain stimuli arising under controlled 
laboratory conditions and reported that high probabilities positively 
influenced response confidence and decision times (Alain & Proteau, 1978, 
1980). More recently, there have been attempts by researchers to move away 
from traditional laboratory-based tasks in an effort to use more sport-specific 
stimuli and tasks (e.g. Belling, Suss, & Ward, 2015; Ward, Ericsson, & 
Williams, 2013). Ward and Williams (2003) presented skilled and less skilled 


players with short (5 sec) video clips involving offensive sequences of play in 


soccer. At the end of each sequence, participants were required to indicate to 
which teammate(s) the player in possession of the ball could pass the ball and 
to then rank the possible options as to their likelihood of occurrence. Skilled 
players were more accurate than less skilled counterparts at this judgment 
task, as determined both by a panel of expert coaches and by what happened 
in the actual match. The key conclusion is that skilled athletes develop high- 
level knowledge structures in memory that enable them to accurately assign a 
hierarchy of probabilities in regard to the likelihood of certain events 
happening at a particular moment in a match. The assumption is that such 


knowledge enables players to make a priori predictions as to what will 


happen next which may then be confirmed or refuted as additional 
information arises in the display. 

A number of attempts to manipulate and study context effects have been 
published in recent years (e.g. Loffing & Hagemann, 2014; Mann, Schaefers, 
& Canal-Bruland, 2014; McRobert, Ward, Eccles, & Williams, 2011; Navia, 
van der Kamp, & Ruiz, 2013). McRobert and colleagues (2011) examined 


how multiple, sequenced exposures to the same bowler in cricket impacted 
on the ability of skilled and less skilled batters to anticipate compared with 
when these same deliveries were presented in a non-sequential, independent 
manner. The additional contextual information presented in the multiple, 
sequenced exposure condition resulted in more accurate anticipation 
judgments and systematic changes in gaze behaviors and think-aloud verbal 
reports relating to thought processes. In particular, with the increase in 
context, skilled batters showed gaze behaviors that were indicative of greater 
reliance on earlier arising cues from the bowler’s movements (i.e. more 
proximal rather than distal cues in the kinematic chain), whereas verbal 
reports suggested an increase in higher-order cognitive processes related to 
prediction and planning rather than lower-order thoughts related to 
monitoring and evaluation. 

Another approach that has been used to examine the role of context in 
anticipation in tennis has been to remove postural information through 
animated displays. In such displays, the only information presented was the 
court markings, the ball, and different color cylinders that enable 


identification of players’ positions (Murphy et al., 2016). Skilled and less 


skilled players were presented with sequences of tennis rallies and asked to 
anticipate where an opponent on the other side of the net would hit the ball 


when trials were occluded at ball—racket contact. In these conditions, players 


were still able to anticipate at levels above chance, with the skilled players 
being more accurate than the less skilled players in both normal and edited 
viewing conditions. In subsequent experiments, the recording of gaze data 
and retrospective verbal reports of thought processes in the animated 
condition showed that the differences in performance across groups were at 
least partially due to the skilled players’ ability to use higher-order cognitive 
information. This ability was verbalized as information relating to court 
positioning, relative movements between players and the ball, shot 
sequencing, rally length, and the angles between the players and the 
baselines. These findings highlight the complex interaction between 
comparatively low-level (e.g. biological motion perception) and high-level 
(e.g. sequencing, probabilities, and court position) cognitive processes during 


anticipation. 


Exploring Interactions between Perceptual-Cognitive Skills 


In preceding sections, a number of different perceptual-cognitive skills have 
been identified. Expert athletes in fast ball sports are able to pick up postural 
cues from an opponent early in an action, are more accurate in detecting 
familiarity and structure or patterns in evolving sequences of play, and have 
greater knowledge of likely event occurrences and probabilities. This 
interaction between processes that may be deemed to involve high- and low- 
level cognitive processes is likely to be dynamic, with the importance of 
these skills varying as a function of numerous constraints on performance 


(Roca & Williams, 2016; Williams, 2009). Figure 34.3 presents a schematic 


illustration of the potential interaction between these different perceptual- 
cognitive skills. Thus far, there have been few attempts to highlight 


systematically how these different skills interact in a single experiment. 
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Figure 34.3 The potential interaction between perceptual-cognitive skills 


and constraints during anticipation judgments (from Murphy, Jackson, & 
Williams, 2015). 


In view of the numerous task constraints that impact the decisions of 
elite athletes in team sports, including where the play is taking place relative 
to their position and role, recent attempts have been made to incorporate 
these subtleties into research paradigms that evaluate anticipation and 
decision making. Roca et al. (2013) developed soccer film sequences from 
the perspective of a central defender, in either a “near” or “far” view. For the 
“near” task, the offensive pattern of play commenced in the defender’s half of 
the field. For the “far” task, the sequence of play started with the ball in the 
opponents’ half of the pitch. Visual gaze data and retrospective verbal reports 
of thoughts were taken as players attempted to anticipate what option the 
opponents would take (i.e. pass, shoot, dribble) and to decide how they would 
respond. Skilled players used different search strategies compared to their 
less skilled counterparts with these differing as a function of the near and far 
task manipulation. In the far task, skilled players employed more fixations of 
shorter durations to more areas of the display compared to less skilled 
players. They also spent more time fixating teammates, opponents, and areas 
of free space in the far task compared to the near task where they spent more 
time fixating on the player in possession of the ball. The less skilled players 
displayed no such differences between task conditions. Gaze data are 


presented in Table 34.1. 


Table 34.1 Mean (SD) number of fixations, number of fixation locations, and 
fixation duration of skilled and less skilled soccer players relative to near and 


far task constraints. Data from Roca et al. (2013). 


Skilled Less skilled 
Search rate Far task Near task Far task Near task 


No. of 13.95 (1.92) 8.17 (1.85) 7.01 (1.23) 5.64 (1.05) 
fixations 


No. of 7.31 (1.37) 4.14 (0.58) 3.53 (0.83) 3.11 (0.64) 
fixation 
locations 


Fixation 332 (61) 598 (205) 745 (174) 877 (173) 
duration 
(ms) 


A novel coding system was used to analyze the verbal reports collected 
during these near and far task decision scenarios, with articulations being 
coded relative to their focus on postural cues, pattern recognition, and 
situational probabilities (Roca et al., 2013). While clear differences were 
evident across skill groups, the most interesting finding was how the 
importance of the three different perceptual-cognitive skills differed across 
the task constraints. In the far task, skilled players verbalized more thought 
processes related to pattern recognition, whereas in the near task, postural 
cues and situational probabilities better defined how skilled players were 


making decisions. These data are presented in Figure 34.4, 
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Figure 34.4 Mean frequency scores (SD) per trial of verbal statements 
relative to the perceptual-cognitive skills referred to by (a) skilled and (b) 


less skilled soccer players under near and far task constraints. 


Data from Roca et al. (2013). 


Context was also manipulated in a simulation-based, anticipation task in 
tennis, where access to varying levels of postural information, shot 
sequencing, and player positioning were interactively studied in one design 
(Cocks, Jackson, Bishop, & Williams, 2016). The authors also studied how 


anxiety influenced the relative importance of the different sources of 


information. With increasing levels of anxiety, tennis players placed greater 
reliance on low-level (such as postural cue usage) rather than high-level (such 
as shot sequencing and court positioning) cognitive processes. Anxiety was 
most detrimental to performance in conditions where only contextual 
information was given and stimulus-driven information was omitted. 

In future, greater efforts should be made to examine the more complex 
nature of perceptual-cognitive expertise by exploring how each of the 
different perceptual-cognitive skills interacts and how other factors like 
context, task, and emotions influence these interactions (Roca & Williams, 
2016). This approach is consistent with recent recommendations by other 


researchers in this area (Cafal-Bruland & Mann, 2015). As Cafial-Bruland 


and Mann (2015) point out, in real performance situations, players will have 


access to various combinations of different sources of contextual information 
and there is a need to investigate how these interact to influence performance. 
Runswick, Roca, Williams, and North (2016) differentiated between 
situation-specific and non-situation-specific context. Situation-specific 
context refers to information leading to the selection and performance of a 
skill that is unique to that event. For example, in baseball, the number of 
innings left in the game, score, position of opposition fielders, and 
sequencing information from a pitcher are unique to each delivery and could 
change from delivery to delivery. In contrast, non-situation-specific context is 
not unique to a single event and includes contextual information that can be 
found at any time during any performance. For example, information about 
past team successes and playing style is independent from the situation of the 
game. Further research is needed to better understand the complex, dynamic, 
and multifaceted nature of perceptual-cognitive processes when performers 


are required to anticipate in sport. 


From Research into Practice: Implications for 
Talent Search and Development 


In the opening sections of this chapter, we highlighted the importance of 
engagement in the right type and quantity of practice, such that specific 
adaptations can occur in order to leverage the unique plasticity and 
adaptability of the human system. These practice activities have been 
considered with reference to the perceptual-cognitive skills that underpin 
anticipation and decision making and how specific, practice-based 
adaptations enable elite athletes to overcome the constraints evident in high- 
performance sporting environments. An implicit thread running throughout 
the chapter so far, is that expertise arises through specific adaptations that are 
comparatively unique to each performance domain. These adaptations are 
dependent on athletes engaging in the appropriate quality and quantity of 
practice and other sport-specific activities. In this final section, we consider 


implications of this empirical work for “talent” search and skill development. 


Searching for the Holy Grail: Can We Predict Sporting Talent? 


The issue of whether future experts may be successfully identified at an early 
age, selected for entry into a systematic sport training program, and attain 
expert performance and later success in the professional sport has been 


debated for many decades (Williams & Reilly, 2000; Vaeyens, Lenoir, 


Williams, & Philippaerts, 2008). Scientists from various disciplines, 
including psychology, physiology, anthropometry, biomechanics, and 
sociology, have attempted to identify early predictors of talent. The “Holy 
Grail” is to identify a test or battery of tests that would predict those 
individuals most likely to succeed in future in the target sport. Generally, 
while measures from each of the above discipline areas are often successful 
in discriminating expert from less expert performers in many sports within 
and across age and skill categories, the predictive utility of such measures as 
part of a long-term talent identification program remains unproven. A 
shortcoming is the relative absence of longitudinal data, particularly using 
comprehensive, multidisciplinary test batteries. The majority of published 
reports are cross-sectional and mono-disciplinary in nature and while these 
papers often highlight expert—novice differences, the proportion of variance 
in performance accounted by many of the variables measured remains 
unknown and the medium- to long-term predictive utility of such measures is 
debatable. 

The challenge in many sports is that there are so many different 
attributes, characteristics, competencies, and skills underpinning performance 
such that a weakness or shortcoming in one of these may be compensated for 


by strengths in other areas. A “compensation phenomenon” exists whereby 


an athlete (e.g. soccer player) may moderate for a weakness (e.g. poor 5 m 
sprint times) by developing other aspects of his/her game (e.g. ability to 
anticipate, technical ability, leadership skills), thereby ensuring that they 
retain “currency” in the competitive environment. Such a phenomenon makes 
it difficult and somewhat unethical to de-select athletes from a talent 
development program based on poor scores on individual components of a 
test battery. Moreover, in team sports, coaches may compensate for a 
weakness in one player (e.g. a lack of pace) by selecting someone to play 
alongside him/her who has compensating attributes. These arguments do not 
imply that there are no known competencies that may be essential to future 
success in a particular sport, neither does it suggest that there are no 
minimum requirements or threshold levels for key components of 
performance; rather they acknowledge the fact that many sports are 
inherently complex and multifaceted. Predicting future success, therefore, is 
difficult because these factors interact in a complex and non-linear manner. 
The scale of the problem may be greatly reduced in sports where there 
are fewer attributes, characteristics, competencies, and skills contributing to 
performance; that is, a higher proportion of the variance between selected and 
non-selected athletes may be explained by a fewer number of predictor 
variables. At one end of the spectrum, sports such as soccer, rugby, 
basketball, and tennis may have a high, if not infinite, number of potential 
predictor variables each accounting for a low proportion of the variance, 
whereas more closed sports such as rowing, cycling, or distance running may 
have fewer predictor variables, that account for more variance, thereby 
greatly reducing the complexity of the prediction process. The scale of the 
problem may be exaggerated by the duration or time period over which any 


prediction must be made. Predicting from a group of children aged 8 years 


which of them may be in an elite group of athletes at 18 years of age is a far 
more difficult task compared to predicting progression from 16 to 18 years of 
age. The shorter the prediction period the more accurate any model is likely 
to be and the greater the variance in performance explained. Finally, the 
socio-cultural popularity of the sport, the extent to which early specialization 
or early engagement in the sport occurs (see Ford & Williams, 2017), the 
number of participants in a sport, and the technical and tactical demands of 
the sport will interact with the viability and predictive utility of any talent 
identification model. Sports which are extremely popular, have mass 
participation figures, and involve early engagement to develop high levels of 
technical and tactical skill (e.g. soccer and tennis) present a much greater 
challenge from a talent identification perspective compared to sports that are 
less popular, where late specialization is the norm and performance is more 
reliant on physical or physiological skills (e.g. rowing, cycling, and many 
Olympic winter sport events such as luge or bob-sled). 

A cost-benefit analysis is needed with any talent identification model 
based on two key questions: what proportion of the variance in performance 
is accounted for by the model and what is the cost of implementing such a 
model? Unfortunately, statistical modeling is rarely undertaken and never 
reported in the literature, therefore making it difficult to evaluate the costs 
and benefits associated with such an approach. Moreover, the judgment as to 
whether such a model presents “value for money” is a subjective one. Would 
a multidisciplinary talent identification model that predicts 35 percent of the 
variance in performance in athletes who progress from one age group to 
another be a good investment of resources? How much variance would need 
to be accounted for to justify the cost? Clearly, presenting answers to such 


questions is not easy and inevitably there is an element of subjectivity 


involved. Supplementary questions would include how much money is 
available to spend and what would be the financial benefits if the model 
presented some moderate predictive utility. Perhaps the main question, 
however, should be: would it would be better to invest some, if not all, of the 
resources set aside for talent identification into the talent development 


process instead? 


Systems, Structures, and Processes: Why Environment Is 
Everything? 
We highlighted earlier research that has shown the importance of deliberate 
practice on the road to performance excellence in sport. While it remains 


debatable as to whether practice alone is sufficient to attain expert 


performance levels in sport (e.g. Hambrick et al., 2014), it is unlikely that any 
athlete will reach an elite level without investing significant time in practice. 
However, it is inevitable that both the quality and quantity of practice interact 
in developing expert athletes. Our view is that more effort should be focused 
on identifying the practice conditions that best promote skill development 
rather than on identifying “talent” per se. In the following, we highlight a 
potential gap that exists between theory and applied practice in coaching. 
Some key findings from the field of motor learning with respect to effective 
practice methods are contrasted with recent work where researchers have 
used video-based, time-use analysis to explore what coaches actually do 


during practice. 


Effective Practice and Instruction 


The distinction between performance and learning is important for motor 
learning research since it has been demonstrated that various interventions 
and practice conditions have differential effects on these two factors. 
Learning cannot be directly observed, but is instead inferred from changes in 
behavior over time and inferences in regard to learning necessitate measures 
of skill retention (i.e. is the change in behavior robust over time?) and 
transfer (i.e. does it transfer to novel situations or variations of the same 
context?) (see Schmidt & Lee, 2013). Performance is the behavior that is 
directly observed during practice and competition. It has been shown that 
performance in practice is better when coaches provide instruction explicitly 
and frequently (either verbally or visually), use practice schedules that 
encourage repetitive, blocked practice of a single skill, and provide copious 
amounts of feedback. However, almost the reverse conditions promote 
effective retention and transfer of skills; that is, coaches should provide only 
the minimal amount of instruction to encourage self-directed practice, they 
should develop random and variable practice conditions that simulate the 
competition setting, and provide prescriptive feedback sparingly so as to 
encourage the learner to become aware of their own errors as well as how to 
correct them (Williams & Hodges, 2005). 

In contrast to the abundance of experimentally driven research on the 
optimal conditions necessary to promote skill learning, relatively few 
researchers have explored what activities coaches have athletes engage in 


during practice (for exceptions, see Cushion, Ford, & Williams, 2012; Ford et 


al., 2010). Ford and colleagues (2010) completed a video-based, time-use 


analysis of 70 coaching sessions involving three different age groups (under 
9, 13, and 15 years) and skill groupings (elite, sub-elite, intermediate level) in 
soccer. Coaches had players engage in physical training, and structured, drill, 
and grid-based practices for the majority (65 percent) of the practice session 
(termed Training Form Activity). Only a minority of the time (35 percent) did 
coaches have athletes engage in activities that were deemed more 
representative of match play such as small-sided games, conditioned games, 
and phase play (activities classified as Playing Form). Moreover, the most 
observed coach behavior was the provision of instruction with periods of 
silence being the least recorded events. 

These data, as well as those reported in other sports, such as cricket 
(Low et al., 2013), suggest a theory versus practice divide in coaching. 
Coaches seem focused on creating conditions that produce the best 
performance during practice rather than on designing activities that foster 
improved retention and transferability of skill. An important issue is the 
extent to which current coach behaviors promote efficiency in regard to skill 
learning. Is practice time well spent and what proportion of current practice 
actively transfers to improvements in performance during competition? A 
challenge would be to develop a transfer quotient whereby practice sessions 
can be evaluated for their efficiency in promoting skill transfer from practice 
to competition. Further research is needed to better inform practitioners and 
coaches as to the conditions that best facilitate the development of expertise 


in athletes. 


Conclusions 


In this chapter, we have reviewed recent research focusing on the topic of 
expertise in sport. In the opening section, we reviewed research carried out 
using the deliberate practice theoretical framework in an effort to identify 
some of the antecedents of expertise in sport. In keeping with research 
focusing on expertise in other domains, it is now well established that expert 
athletes accumulate significant hours in practice within their chosen sport. 
The debate continues as to the relative importance of engagement in play vs. 
practice vs. competition, as well as the contribution of activity in the sport 
itself compared with engagement in other sports on the road to excellence. 
Yet, regardless of the nuances of the debate, the need to participate in large 
amounts of high quality, sport-specific activity is established as a key 
precursor to expertise. In the second section, we highlighted some of 
adaptations in perceptual-cognitive skill that arise through prolonged 
engagement in practice and other domain-specific activities. These 
adaptations were seen as largely specific to the nature and types of practice 
activities engaged in by athletes and highlight both the demands of each sport 
and the considerable plasticity of the human system. Nonetheless, we 
acknowledged that some “low-level” transfer is possible between sports that 
have comparable elements or similarities in processing. We then focused 
exclusively on the key perceptual-cognitive skills that underpin anticipation 
and decision making in fast ball sports. We showed how various perceptual- 
cognitive skills interact dynamically to circumvent human limitations in 


memory and information processing, enabling athletes to cope with the 


demands of high-performance sport. In the final section, we discussed the 
implications of research on deliberate practice and on the development of 
anticipation for the identification and selection of future elite athletes. We 
discussed some of the challenges involved in talent identification programs 
and debated the practical utility and costs and benefits of investing resources 
in this area. We considered the importance of developing effective support 
systems and structures and in investing in coach education programs to help 
ensure that practice is effective with performance improvements transferring 
to the competitive environment. Our intention was to highlight how empirical 
research on expertise can help facilitate future generations of elite athletes, as 


well as experts in other professional domains. 
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Introduction and Overview 


To move successfully in a dynamic world we need to be not only aware of 
the current but also the future location and movement trajectories of 
ourselves, our limbs, and the elements of the environment with which we 
interact. The ability of the brain to function in a predictive or anticipatory 
fashion is therefore clearly integral to skilled performance. 

An association between superior anticipation and expert performance 
has long been hypothesized. The famous British psychologist, Sir Frederic 


Bartlett, observed, in an early treatise on skill, that: 


There is one characteristic which crops up over and over again in 
descriptions of expert skilled performance. The operator is said to have 
“all the time in the world to do what he wants”. This has nothing to do 
with the absolute speed of the constituent movements, bodily or mental. 
These may be incredibly quick, or they may be leisurely and slow. What 
is impressive is the absence of any appearance of hurry in the whole 


operation. 


(Bartlett, 1947, p. 836) 


This chapter will examine the role that predictive and anticipatory 
behavior plays in the expert performance of a variety of different movement 
tasks. The expert advantage will be described across situations that require 
prediction for interception as well as prediction of the motion of others. The 
growing body of evidence in relation to the nature of expert anticipation will 


be used to consider some of the key challenges that need to be overcome in 


order to systematically “fast track” the learning of the predictive skills of 


experts. 


Creating Time by Anticipating the Actions of 
Others 


Evidence of an Expert Advantage in Anticipatory Behavior during 
Interceptive Tasks 
In many work and sport tasks the time constraints under which movements 
must be planned and executed are so severe as to necessitate the 
commencement of these movements on the basis of partial or probabilistic 
information. Chronometric analyses of various sports tasks, such as hitting a 
baseball or cricket ball, returning a stroke in tennis, or avoiding the attacking 
move of a boxer or martial artist, indicate that there is often insufficient time 
available to wait until seeing the confirmatory information of the approaching 
ball or the commencement of the opponent’s attack to use this as the basis for 
triggering an appropriate interceptive or avoidance response (Glencross & 
Cibich, 1977; Miller & Shay, 1964). Given that the normal reaction time 


delays observed in laboratory studies of human performance appear to be 


largely preserved within “real-world” tasks such as driving, catching, or 
hitting (e.g. see McLeod, 1987), and that the movement times needed for 
responding frequently approach or exceed the total time available for a 


particular action sequence (Whiting, 1969), it is generally conceded that the 


triggering event for movement responses in such circumstances must come 
from advance information. By way of example, a goalkeeper in soccer takes 
some 600—1000 ms to dive from the center of the goal to either corner of the 
goal while the penalty kick itself make take as little as 400 ms to reach the 
goal, making initiation of the goalkeeper’s movement prior to the penalty 
taker’s strike of the ball imperative to successful interception (Dicks, Uehara, 
& Lima, 2011). In sports tasks, this advance information upon which 
movement responses are based is thought to come primarily from information 


contained within the preparatory movements of the opponent(s) (e.g. the 


movements of the penalty taker that precede contact with the ball) although 
contextual information (e.g. knowledge of the penalty taker’s prior preferred 
kick options) is also likely important (Canal-Bruland & Mann, 2015). 

To test the notion that there is useful advance information for response 
planning available from the complex movement patterns of other people 
(opponents in the sport context), and that sensitivity to this information may 
be a hallmark of expertise, a number of laboratory studies of anticipation 
using domain-specific stimuli have been developed. Beginning as early as the 
work of Enberg (1968) and Jones and Miles (1978), film or video stimuli 
have been created that permit the vision of the opponent’s action to be 
systematically occluded either before, at, or after critical events such as the 
beginning of ball flight. Performances better than chance in discrete choice 
tasks (such as, is the ball being hit to the forehand or backhand side?) 
performed in the absence of ball flight information are taken as evidence of 
sensitivity to advance information that can support anticipatory responding. 
A compelling body of evidence from across a range of different performance 
domains points to both the availability of such information to support better- 
than-guessing responses and the greater sensitivity of domain-specific experts 
to such information (see Abernethy, Maxwell, Jackson, & Masters, 2007 for a 
review). 

While the potential use of advance information from the movement 
pattern of others to help explain the superior performance of experts (and the 
underlying basis for the phenomenological impression they create of being 
unhurried) has been confirmed from laboratory studies of perception these 
studies do not in themselves prove that such information is in fact used in 
situ. Unlike the laboratory situation where response modes are typically quite 


simple (often simple verbal, pencil-and-paper, or finger press responses) and 


without stringent time constraints for responding, the natural performance 
environment requires these complex perceptual judgments to be performed 
under extreme temporal and spatial constraints alongside concurrent 
movement preparation requirements. Regan (1997), for instance, noted that in 
hitting a baseball temporal accuracy within + 9 ms and spatial accuracy 
within + 0.05 in (1.27 cm) is required to strike a 90 mph (40.2 ms‘!) fastball. 
Indeed there is some suggestion that the kind of responses required in typical 
laboratory tasks may use a fundamentally different visual pathway than that 
used in “real” interceptive activities (Milner & Goodale, 1995; van der 
Kamp, Rivas, van Doorn, & Savelsbergh, 2008). 

Empirical studies of experts performing in actual competition indicate 
the earlier initiation of appropriate movement responses (e.g. the first step in 
the correct direction for responding), consistent with superior pick-up and 
utilization of advance information (Abernethy, Gill, Parks, & Packer, 2001; 
Hennemann & Keller, 1983; James, Caudrelier, & Murray, 2006). Studies in 
situ, where real movement responses are required and occlusion of vision is 
provided using remotely triggered liquid-crystal goggles (Milgram, 1987), 
support the key findings from laboratory studies, namely, the greater 
capability of domain-specific experts to both pick up information at any point 
during the event sequence and use this information to advance their 
movement response relative to that of non-experts (Miiller & Abernethy, 
2006; Starkes, Edwards, Dissanayake, & Dunn, 1995). Indeed the more the 


perception and action requirements of the natural performance setting are 


replicated experimentally, the more pronounced the expert advantage in 
anticipation becomes (Mann, Abernethy, & Farrow, 2010). The use of 
representative designs appears important for the full capture of expertise in 


anticipation (Pinder, Davids, Renshaw, & Araujo, 2011). 


Evidence of Differential Use of Perceptual Information 


The demonstration of a consistent expert advantage on anticipatory tasks has 
promoted systematic experimental search to identify more specifically the 
sources of advance information to which experts are more selectively attuned 
than non-experts, and which, in turn, might provide an explanation of the 
anticipatory advantage enjoyed by experts. One approach (temporal 
occlusion) involves systematically varying when the visual display is 
occluded so as to create viewing conditions with more or less of the whole 
event sequence visible. Studies using progressive temporal occlusion have 
typically revealed that not only are experts able to extract more information 
(and hence make better predictions) from any time window of advance 
information but there are some specific time windows early in event 
sequences, such as those early in the hitting actions of opposing players in 
racquet sports, where only experts are able to extract information to support 
better-than-chance predictions (e.g., Farrow, Abernethy, & Jackson, 2005). 
Concurrent use of spatial occlusion — a technique where vision of some 
particular cues is occluded throughout the event sequence — provides a means 
of drawing more direct inference as to which features of the opponent’s 
movement patterns provide information as to their intentions and which 
features can consequently help in the early preparation of an appropriate 
movement response. In an early laboratory example of the use of both 
temporal and spatial occlusion techniques Abernethy and Russell (1987a) 
demonstrated dependence on early-occurring information from the motion of 
the arm holding the racquet as a key point of differentiation between the 


perceptual skills of expert and non-expert badminton players. Displays 


showing only the motion of the bowler’s hand holding the ball in cricket are 
sufficient to permit expert batters to make better-than-chance predictions of 
the type of ball being bowled whereas non-experts show no such capability 
(Miiller, Abernethy, & Farrow, 2006). In situ examples of spatial occlusion 


methods are now emerging (e.g. Panchuk & Vickers, 2009) and revealing 


similar general characteristics of the expert advantage. 

Both occlusion techniques provide behavioral evidence of the predictive 
capabilities of performers when display information is restricted to only a 
limited set of features, hence providing a relatively direct measure of the 
information pick-up capability of performers of different skill levels and of 
the potential value of different features/cues for supporting anticipation. A 
less direct approach is to imply differences in the perceptual strategies of 
expert and non-expert performers by either monitoring their gaze behavior as 
they view their opponent’s unfolding action or asking them to report verbally 
on what particular cues they direct their attention toward in order to anticipate 
their opponent’s action. Studies measuring the eye movement (visual search) 
patterns of experts have frequently, but not always, revealed differences in 
the location and frequency of the ocular fixations of experts and non-experts, 
with a greater frequency of fixations to those elements of the emerging 
movement pattern that contain critical cues, as identified through spatial 
occlusion methods. The challenge with a singular reliance on gaze 
measurement to understand expert perception is that gaze on a particular area 
of the display need not necessarily be coincident with either attention or 
information pick-up and consequently expert—novice differences in advance 
information pick-up can occur even when the visual search patterns are 
indistinguishable (e.g. Abernethy & Russell, 1987b). More recent studies 


using gaze-contingent methodologies to force alignment between attention 


and gaze reinforce this key observation that differences in the predictive 
performance of domain-specific experts and novices can occur regardless of 
whether critical information is foveated or not (Ryu, Abernethy, Mann, 
Poolton, & Gorman, 2013). Verbal report protocols appear to be useful for 
probing conscious strategies developed by participants of different skill levels 
(Ericsson & Simon, 1993) but are unsuitable for use in situations of the type 
that typically occur in domains which are time-constrained and characterized 
by uncertainty and dynamic display changes, where control is either 
automatic or below the level of consciousness (Nisbett & Wilson, 1977). In 
circumstances where adaptive, anticipatory responding is needed, verbal 
reports from performers as to the cues that they use for anticipation result in 
either similar insights from experts and non-experts or introspections which 
are demonstrably inaccurate (e.g. Bahill & LaRitz, 1984). 

The consistent finding from the temporal and spatial occlusion studies, 
mirrored in most cases by gaze behavior data, is that experts are more attuned 
to earlier occurring, more biomechanically proximal sources of information 
in the action of their opponents than are non-experts. The notion that there is 
a tight coupling between expert anticipatory skill and sensitivity to the 
kinematics of the unfolding movement pattern that they are predicting can, 
and has, been examined experimentally. Point-light displays (Johansson, 
1973) provide a means of reducing complex visual displays to simply raw 
kinematic information. Such displays can therefore provide a means of both 
determining empirically the extent to which the expert advantage relates to 
superior pick-up of kinematic (as opposed to textural or featural) information 
and manipulating key kinematic features to see if these can influence or bias 
predictions about the evolving movement. It has been demonstrated that the 


bulk of the expert advantage is preserved when displays are reduced to 


simply points of light corresponding to the motion of the key 
joints/biomechanical segments of the opponent, demonstrating the primacy of 
kinematic information to expert anticipation (e.g. Abernethy et al., 2001). 
Selective presentation of points of light representing key biomechanical 
segments show an expert sensitivity to linked segment information that is not 
apparent for non-experts, providing further support for the notion that skilled 
pick-up of information for anticipation is tightly coupled to the biomechanics 
constraints and evolution of the action being predicted (e.g. Abernethy & 
Zawi, 2007). Identification of the biomechanical differences between actions 
with different intentions (e.g. the biomechanics of a tennis stroke hit down- 
the-line as opposed to cross-court) is also providing a fertile means of 
seeking to identify the subtle perceptual cues to which expert performers may 
be attuned (e.g. Huys, Smeeton, Hodges, Beek, & Williams, 2008). 

For a more detailed description of these different methodological 
approaches to examining anticipatory expertise and the mechanisms 


underpinning it see Williams and Abernethy (2012). 


Explanations of the Evolution of the Expert Advantage 


There are at least two competing explanations as to the evolution of the 
expert advantage in using perceptual information to guide effective 


anticipatory responding. The traditional view, that stems from perceptual 


learning theorists (e.g. Gibson, 1969; Goldstone, 1998), is that the repeated 
exposure to domain-specific stimulus patterns gives rise to the greater 
capability (either consciously or otherwise) to differentiate stimulus patterns, 
to respond to redundancies in stimulus patterns, and to recognize earlier the 
beginnings of a particular stimulus sequence. It is through such early 
recognition of perceptual sequences that a given stimulus event can become 
consistently coupled to a more advanced element of the appropriate 
movement response than is possible for a non-expert (Miiller & Abernethy, 
2012). 

An alternative view, grounded in the common coding view of movement 


perception and production (e.g. Prinz, 1997), is that the expert’s perceptual 


advantage is a direct consequence of their expertise in producing the very 
same class of movements that they are required to anticipate. The common 
coding view maintains that the same cognitive representations are used for 
perceiving the intention of an action as for producing the action and that in 
observing an action our brains simulate producing the same action, with this 
simulation enacted through the mirror neuron network in the inferior parietal 


cortex and premotor cortex (Buccino, Binkofski, & Riggio, 2004). Evidence 


supporting the common coding view comes from studies showing that people 
are more accurate in predicting their own actions when they view them than 


the actions of others (Knoblich & Flach, 2001) and that the mirror neuron 


system is more active when people view actions that they can produce than 
either unfamiliar actions or ones they cannot themselves execute (e.g. Calvo- 
Merino, Glaser, Grézes, Passingham, & Haggard, 2005). 

The empirical evidence to support the common coding explanation of 
the expert advantage in anticipation is currently mixed. While imaging 
studies (e.g. Wright & Jackson, 2007; Wright, Bishop, Jackson, & Abernethy, 
2010; Wright, Bishop, Jackson, & Abernethy, 2011) suggest that the mirror 


neuron system is activated differently in anticipation tasks for experts and 
non-experts there is little compelling parallel behavioral evidence to indicate 
that this is the root cause of the expert advantage. Attempts to test the motor 
familiarity hypothesis that skilled players should be better able to anticipate 
action outcomes when viewing images of themselves performing a domain- 
specific action than when viewing a teammate or a stranger (Jackson, van der 
Kamp, & Abernethy, 2008) have failed to reveal the expected effect. Further 
experimental evidence is needed to assess how the extent of both perceptual 
and motor learning may contribute to anticipatory skill (e.g. Cafal-Bruland, 


van der Kamp, & van Kesteren, 2009). 


Creating Time by the Anticipatory Encoding 
of Movement Pattern Information 


Evidence of an Expert Advantage in Movement Pattern Encoding 


Patterns are a ubiquitous feature of our environment. Whether playing a game 
of chess, basketball, or driving a car there is typically structured perceptual 
information available that the performer can use to guide their movements. 
Importantly, the manner in which experts use the information contained 
within structured patterns is fundamentally different from the non-experts. 
Specifically, expert motor performers demonstrate a superior capacity to 
interpret the different types of visual information available for the production 
of action. The expert advantage in pattern encoding was _ initially 


demonstrated in the domain of chess (Chase & Simon, 1973; de Groot, 1965). 


This section will focus on evidence accrued from the domain of team sports, 
as the patterns formed by the relative locations of players in sports such as 
basketball and football are typically dynamic and fast-paced ensuring a 
premium is placed on anticipation (Allard, Graham, & Paarsalu, 1980). 

The most prevalent experimental approach utilized in the sport expertise 
literature to understand pattern encoding has been the pattern recall task. In 
original studies of pattern recall performers were presented with a brief slide 
(approx. 5 sec exposure) of a structured or unstructured piece of game-play. 
For example, within the game of basketball, a structured pattern would 
usually contain attacking or defensive patterns of play whereas an 
unstructured pattern may relate to a time-out or post-game mingling of 
players from different teams. Importantly, both slide types would contain 
similar numbers of individual elements (or players) to recall. In the structured 
slides the elements conformed to a pattern meaningful to expert performers 
whereas in the unstructured slides the elements (player positions) were 


completely random. Having viewed the slide, participants were required to 


recall the final locations of all the players when the slide was occluded. 
Typically, expert performers have been found to recall more information 
from structured patterns commonly encountered within their sport. However, 
this advantage is lost when an unstructured pattern is displayed — the 
performance of experts and novices being indistinguishable under such 
circumstances (Allard et al., 1980; Helsen & Starkes, 1999; Starkes & 


Deakin, 1984; Williams & Davids, 1995). In more recent times representative 


test stimuli in the form of dynamic video images have typically replaced the 
static and/or schematic images initially used. This methodological change has 
better preserved the normal dynamic environment that is characteristic of the 
actual performance setting (Borgeaud & Abernethy, 1987; Gorman, 
Abernethy, & Farrow, 2011; Williams, Davids, Burwitz, & Williams, 1993). 


Closely related to pattern recall is the task of pattern recognition. The 


key difference between a pattern recall and recognition task is that instead of 
being asked to recall where all elements (players) are in a pattern the 
performer in a recognition task is simply asked to recognize whether a 
particular pattern has been seen previously (North, Williams, Hodges, Ward, 
& Ericsson, 2009; Williams & Davids, 1995), The findings from such an 
approach have generally been analogous to those from pattern recall with 
experts more accurate in their recognition of the patterns previously seen than 
non-experts. However, one key difference in the findings between the two 
tasks is that the expert performers do not always regress to non-expert 
performance levels when unstructured patterns are presented (North et al., 
2009; Williams & Davids, 1995). 

It has been suggested that the pattern recognition procedure probes a 
performer’s ability to recognize rapidly what is developing in terms of an 


offensive or defensive pattern of play, and thus take advantage of this 


awareness of what a teammate or opponent is likely to do next (Starkes, 
Allard, Lindley, & O’Reilly, 1994). Such reasoning highlights the potential 
functional value of the underpinning mechanisms of pattern recall and 
recognition in the natural performance setting. Notwithstanding this, only a 
limited amount of research has considered whether pattern recall/recognition 
and anticipation and decision making are separate processes (with the former 
preceding the latter) or whether effective anticipation and decision making 
are simply variants of pattern recall. This issue has both theoretical and 
practical significance as pattern recall and anticipation/decision making are 
characterized in many models of cognition and information processing as 
separate perceptual processes; this, in turn, suggests they may need to be 
practiced in different ways in order to develop expertise. Based on the 
collective evidence to date from a range of domains it appears that pattern 
recall is measuring secondary processes to decision making skill (Ericsson, 
Patel, & Kintsch, 2000; Gorman, Abernethy, & Farrow, 2013; Ward, 
Williams, & Hancock, 2006); however, the strength of this relationship may 
be mediated by the specific constraints of the task being performed (such as 
the degree of structure within the pattern information presented; Farrow, 
McCrae, Gross, & Abernethy, 2010). 


Evidence of Superior Anticipatory Encoding of Patterns 


De Groot’s (1965) investigation of chess expertise demonstrated a tendency 
for the master player to reproduce the chess pieces in a position on the board 
where they were likely to be moved at some point in the near future, as 
opposed to placing the pieces in the location that was displayed during the 
actual recall task. In more recent times this observation of potential 
anticipatory encoding has been examined in more detail using some clever 
experimental manipulations of the traditional pattern recognition paradigm. 
Didierjean and Marmeéche (2005) required basketball players of differing 
levels of expertise to differentiate between pairs of schematic basketball play 
patterns. However, the two schematic patterns in each pair were presented in 
three different ways in an attempt to understand the encoding processes of the 
performers. One method presented the two patterns in the usual chronological 
order of a game; the second presented the two patterns in reverse order; while 
the third method presented two identical patterns. A second experimental task 
used the same test stimuli but required participants to memorize a set of 
patterns before attempting to identify those same configurations from 
amongst a set of new patterns during a recognition phase. The new patterns 
included possible subsequent states and possible earlier states of the 
memorized patterns. The results from both experimental tasks showed that 
experienced basketball players tended to be less accurate when attempting to 
differentiate between patterns that were presented in the normal progression 
of a game. This result was attributed to the experienced players anticipating 
the movements of the players depicted within the patterns and thus encoding 


the first presented image as a possible subsequent state, rather than encoding 


the image in its actual form. Such encoding made it difficult for the 
experienced players to differentiate between patterns when they were 
presented in the usual chronological order of a game. 

More recently Gorman et al. (2011) extended the work of Didierjean and 
Marmeéche (2005) by utilizing dynamic video images from an actual 
basketball game rather than schematics. The results were similar to the 
original in supporting the proposition that, when perceiving structured 
patterns from their domain of expertise, experts appear to apply their strategic 
knowledge of such situations in a way that leads to greater anticipatory 
encoding of the viewed pattern. 

The expert’s anticipatory encoding has also been demonstrated in a 
variant of the pattern recall paradigm. Gorman, Abermethy, and Farrow 
(2012) extended the typical recall paradigm to also examine the magnitude of 
anticipatory encoding. Basketball players of differing expertise levels were 
required to complete a typical pattern recall task; however, the measurement 
approach compared recall performance not only to the target pattern at the 
point of occlusion but to a series of subsequent pattern states that occurred 
immediately after the target pattern (at 40 ms increments). This measurement 
approach permitted estimation of the temporal magnitude of the anticipatory 
effect. The distance between the participants’ entered pattern elements and 
the actual pattern elements were calculated to produce a measure of recall 
error. The results demonstrated that experts’ recall of the pattern was more 
closely aligned to patterns that occurred on average 170 ms beyond the state 
of play that was occluded for recall. This temporal advance was significantly 
further into the future than that observed for novices. These findings suggest 
experts are anticipating the evolution of the presented patterns. Interestingly, 


the data also suggest that more traditional pattern recall measurement 


approaches may underestimate the actual expert advantage in such tasks 
(Gorman et al., 2012, 2013). 


Explanations of the Expert Advantage in Anticipatory Encoding 


The term representational momentum has been coined to explain the 
anticipatory encoding displayed by experts that is reduced or largely absent in 
those without the same level of domain-specific experience. All observers 
extrapolate the temporal features of the stimuli they are observing to some 
degree. Observers tend to overestimate the final location of the stimuli if it 
portrays a greater degree of implied or actual velocity (Finke, Freyd, & Shyi, 
1986; Freyd & Finke, 1985). It is proposed that this phenomenon occurs 


because observers anticipate the path of the object being viewed and 
remember that object by integrating predicted motion with perceptions of its 
implied motion (Didierjean & Marméche, 2005; Finke et al., 1986). 


In the case of patterns that occur within the sport domain it has been 


demonstrated that the relational distances and relative motions between the 
players are critical sources of information utilized by the expert performers to 
extract perceptual information to facilitate pattern recognition (Williams, 


Hodges, North, & Barton, 2006). Furthermore, the essential information 


needed to guide the pattern recognition process appears to be delimited to 
specific relational information between only a few key players (e.g. central 
attackers and their immediate opponents). In contrast, less skilled players 
depend more on superficial elements such as the team’s uniform color to 
guide their pattern perception (North, Ward, Ericsson, & Williams, 2011; 
Williams et al., 2006). Extraction of motion information and the temporal 
relationship between key elements, both referenced to complex memory 


representations, appears fundamental to expert pattern recognition, 


suggesting that an interactive form of encoding is in operation (see Dittrich, 
1999). 

Of particular interest within expertise research is whether these 
anticipatory processes can be moderated by the observer’s expectations, 
personal experience, and knowledge base. Gorman et al. (2011) examined 
this issue by displaying pairs of basketball patterns (presented 
chronologically and in reverse order) to expert, recreational, and novice 
basketball players in addition to a group of skilled soccer players with little to 
no basketball experience. Consistent with previous research, the experts 
struggled to differentiate between pairs that were displayed chronologically 
rather than in reverse order when the first image in the pair was presented as 
a static schematic or a moving video extracted from an actual basketball 
game. The inclusion of the group of soccer players was designed to examine 
whether the representational momentum effect would transfer to other 
domains. The results did not reveal any such effect suggesting the soccer 
players lacked sufficient knowledge of the typical basketball game play 
dynamics to be able to accurately predict the next likely state of play. 
Coupled with other research (e.g. Gorman et al. 2012; North et al., 2011) it 
can be concluded that representational momentum is a likely underpinning 
mechanism of the anticipatory encoding skills displayed by experts and is 
enhanced by the rich domain-specific experience such performers have 


accrued. 


Creating Time by Predictive Gaze Behavior 


Relationship of Eye Movements to Anticipatory and Interceptive 
Skill 

The examination of visual gaze provides a useful means of examining the 
anticipatory strategies used by expert performers. As highlighted earlier, 
skilled athletes can anticipate the actions of their opponents at an earlier 
moment than others, and that advantage can sometimes be attributed to the 
gaze of the skilled performers being directed toward body segments that 
move earlier in a kinematic action sequence. The expert advantage in 
anticipation is also observable when skilled athletes intercept fast-moving 
targets such as a tennis or hockey ball. Because of the significant 
sensorimotor delay required for a movement to be updated on the basis of 
visual information (150—200 ms; Georgopoulos, Kalaska, & Massey, 1981), 
it is generally accepted that a degree of prediction is required to anticipate 
where the object will be at the moment it is hit, so that the hitting implement 
can be there in time to intercept it (e.g. Diaz, Phillips, & Fajen, 2009). Gaze- 
tracking systems that record the direction of gaze when performing 
interceptive tasks confirm that prediction is a key component of how experts 
overcome their neural delays to successfully intercept the target because gaze 
is frequently directed to where the target is expected to be in the future. 
Ripoll (1989) showed that the gaze of five expert table tennis players moved 
in advance of the ball to where it was expected to be when it moved over the 
net and/or when it bounced, rather than tracking the ball throughout its flight. 
The pre-positioning of gaze toward the future positions of the ball was found 
to be a key characteristic of the gaze of the skilled players. 

Given the propensity for gaze to act in an anticipatory fashion during 


interception, it might not come as a surprise that expert performers act in a 


more anticipatory manner than do non-experts. In their landmark study in 
Nature Neuroscience, Land and McLeod (2000) showed that earlier 
predictive eye movements differentiate a good from a poor cricket batsman 
when hitting a ball. Land and McLeod recorded the direction of gaze of three 
cricket batsmen, each of a different skill level (professional, good amateur, 
and poor amateur), and demonstrated that the batsmen did not look toward 
the ball for most of the time from ball-release to bounce, but instead 
produced a predictive saccade (a fast eye movement) to where they expected 
the ball to bounce. Crucially, Land and McLeod reported that the better 
batsmen produced earlier saccades than the weak amateur, demonstrating 
their superior capacity to anticipate where the ball would be in the future. 

It has been claimed that the anticipatory saccades of the type performed 
by skilled table tennis and cricket players are necessary because the ball 
moves too fast to be accurately tracked by the eyes (Bahill & LaRitz, 1984; 
Ripoll & Fleurance, 1988). However, a recent study by Mann, Spratford, and 


Abernethy (2013) has shown otherwise. They tracked the eye movements of 
two of the world’s best cricket batsmen (and two good amateurs as controls) 
and showed that the elite batsmen largely coupled the movement of their 
head to the ball, meaning that they could have accurately directed their gaze 
toward the ball with little or no eye movements at all. The batsmen rotated 
their head downwards at the same angular velocity as the ball, ensuring that 
the eyes could have been directed toward the ball if kept still and only the 
head was moved. Instead, the elite batsmen used not one but two anticipatory 
saccades on most trials, one to the bounce and the other initiated immediately 
after the bounce, to predict where the batsman expected bat—ball contact to 


occur. This highlights the anticipatory nature in which the elite batsmen 


approached the task, with more time spent with gaze directed in advance of 
the ball than toward it. 

Anticipatory eye movements produced during interceptive tasks are 
made possible by the integration of real-time information with what has been 
learned in the past. Diaz and colleagues (Diaz, Cooper, Rothkopf, & Hayhoe, 
2013) tested the eye movements of novice racquetball players who were 
asked to “hit” balls that bounced toward them in a virtual environment. The 
novices produced the predictive saccades toward the bounce found in other 
studies although Diaz et al. found that the saccades relocated gaze to a 
position that was just beyond the bounce point. More importantly, though, the 
timing and the size of the saccades changed commensurate with a 
manipulation to the elasticity of the ball. When the elasticity increased (i.e. 
the ball became “bouncier”), the saccades occurred earlier, and were larger, 
showing that even novices integrated their existing knowledge about the ball 
with real-time information about the ball flight. Remarkably, the saccades 
were adapted to ensure a constant time delay from when the ball bounced to 
when it was realigned with gaze (~170 ms), irrespective of changes in the 
motion trajectory. This suggests that even the less skilled participants taking 
part in this novel task were capable of using an experience-based model that 
integrated past and real-time information to predict how the position of a 
moving target would change over time (see also Hayhoe, McKinney, Chajka, 
& Pelz, 2012). 


Contributions of Central vs. Peripheral Vision 


Studies of gaze reveal an expert advantage for information pick-up when 
using central vision, and recent evidence shows that a similar advantage 
exists when experts use their peripheral visual field. A range of experimental 
techniques have been employed to selectively isolate the central and 
peripheral areas of the visual field to establish their putative roles across a 
variety of perceptual and motor tasks. In gaze-contingent displays, for 
example, information can be selectively presented to either central or 
peripheral vision in a dynamic way by monitoring and predicting future line 
of gaze and then placing either a mask or window around that part of the 
visual display to which gaze is directed (e.g. Rayner, 1975). The 
overwhelming conclusion from these studies has been that skilled performers 
facilitate superior performance by making better use of their peripheral visual 
field in order to support and complement the role of central vision. 

In the context of anticipation, a key role for peripheral vision is to 
monitor the performance of real-time tasks so that central vision can be 
redeployed toward information that guides performance in the future. For 
instance when walking, peripheral vision is used to regulate the trajectory of 


the lower limbs to guide foot placement (Paillard & Amblard, 1985) and to 


avoid obstacles (Marigold, Weerdesteyn, Patla, & Duysens, 2007) so that 


central vision can monitor the terrain to be walked across in the future 


(Marigold & Patla, 2007). Similarly when driving, peripheral vision regulates 


lane maintenance (Mourant & Rockwell, 1970) and attends to potentially 


distracting information within the vehicle (e.g. speedometer and other 


accessories; Summala, Nieminen, & Punto, 1996), while central vision is 


used to look ahead of the vehicle. Crucially, this allows experienced drivers 
to look further ahead of the vehicle than less experienced drivers (Land & 
Horwood, 1995), permitting more effective monitoring of oncoming traffic 
and more time to prepare to respond if a potentially dangerous event is 
detected. 

However, more than simply playing a supporting role in monitoring 
real-time performance, peripheral vision itself has its own capacity for 
anticipation and decision making. When reading, peripheral vision is used to 
anticipate the best position for the next fixation to be located (Rayner, Inhoff, 


Morrison, Slowiaczek, & Bertera, 1981), with experienced readers better able 


to use their peripheral vision to place their next fixation further in advance of 
where a less experienced reader would. Expert athletes also have a 
perceptual-cognitive advantage when using their peripheral vision. When 
anticipating the outcome of an opposing team’s attack, the predictions of 
skilled volleyball players are worse if either central or peripheral vision is 


removed (Schorer, Rienhoff, Fischer, & Baker, 2013), suggesting that both 


areas have an active role to play in anticipation. Crucially, when using 
peripheral vision only, the anticipatory performance of the skilled players 
remains superior to that of less skilled volleyball players. Similarly, the 
decision making performance of skilled basketball players remains superior 
to less skilled players when information is available only to the peripheral 
segment of the visual field (Ryu, Abernethy, Mann, & Poolton, 2015; Ryu et 
al., 2013), highlighting the skilled players’ capacity to make perceptual 
judgments using information picked up solely by peripheral vision. 
Surprisingly, inexperienced basketball players can learn to improve 
information pick-up when using their peripheral visual field by training with 


a blurred periphery, presumably because it develops improvements in overall 


information pick-up (when using central vision) that can generalize across the 
whole of the visual field (Ryu, Mann, Abernethy, & Poolton, 2016). 


Can the Ability to Anticipate be Trained? 


Studies of Practice History — Anticipatory Skill Correlates 


While there has been a significant amount of research completed on both 
anticipatory skill and the practice and developmental histories of expert 
performers there has only recently been a focus on integrating these two lines 
of evidence as it pertains to the development of anticipation. Of particular 
interest is the influence of practice type, quantity, and quality on anticipatory 
skill development and the ensuing degree of specificity required for 
successful transfer. 

Weissensteiner, Abernethy, Farrow, and Miiller (2008) cross-sectionally 
examined the anticipatory skill of cricket batsmen (skilled and less skilled 
U15, U20, adult batsmen) and whether their anticipatory performance was 
associated with their developmental history profiles (i.e. accumulated hours 
spent in organized and unorganized sport). Across all three age groups, the 
highly skilled players self-reported having accumulated two to three times 
more hours of organized cricket experience than their age-matched controls. 
Consistent with the transition of the older skilled players into the investment 
stage of sports participation (e.g. Coté & Hay, 2002) the total number of 
hours accumulated in unorganized cricket activity and invasion sport 
participation did not differ amongst the older age groups, despite the U15 
highly skilled players having amassed more hours than their age-matched 
control group. Furthermore highly skilled players did not participate in a 
greater range of sports (either organized or unorganized). It was suggested 
that the skilled players’ superior anticipatory skills were closely associated 
with having accrued significantly greater amounts of cricket-specific 
experience than their less skilled counterparts. However, surprisingly, the 


variance in anticipatory performance explained by accrued practice hours, 


independent of the variance simply explicable by age, was only 11-13 
percent. One potential reason for this effect was addressed in a related study 
by Ford, Low, McRobert, and Williams (2010). Cricket batsmen were 
categorized according to their anticipatory skills test performance and 
assigned to a high or low performing group. It was demonstrated that the 
high-performing anticipation group had accumulated more of their hours in 
structured activity in batting practice and match-like batting practice, when 
compared with their low-performing peers, who accumulated more of their 
hours in less game-like practice activities. This reinforces the need to better 
detail the micro-structure of practice as it relates to anticipatory skill rather 
than simply just measuring the amount of hours of experience in order to 
generate a more thorough understanding of anticipatory skill development. 
Similar research has also been completed in invasion game team sports 
such as football. Berry, Abernethy, and Coté (2008) investigated Australian 
football and more recently Roca, Williams, and Ford (2012) have 
investigated soccer. Unlike the cricket observations above, the football code 
research has revealed a number of developmental nuances that may be either 
sport or culturally dependent. Berry et al. (2008) used coach ratings to 
categorize 32 elite level players as either expert or less expert decision- 
makers. No differences were evident between groups in the number of 
structured activities participated in throughout their development but the 
accumulated number of hours was greater for the expert group. Furthermore, 
the expert group invested more time in structured invasion practice and the 
number of hours spent in participating in invasion sports discriminated the 
two groups. Consistent with the Developmental Model of sports participation 
and one of its underpinning concepts of deliberate play (Coté, Baker, & 


Abernethy, 2007), the expert group demonstrated greater investment in the 


time committed to deliberate play by 12 years of age. Such findings suggest 
that while clearly structured practice is important, it is not the only factor. 
There appears to be a degree of positive transfer of anticipatory aspects of 
performance amongst invasion games that share common perceptual or 
decision making requirements (see also Abernethy, Baker, & Coté, 2005; 
Smeeton, Ward, & Williams, 2004). Somewhat different findings have 
emerged from the sport of soccer (in the UK). Roca et al. (2012) reported that 
the average hours per year in soccer-specific play activity during childhood 
was the strongest predictor of decision making performance. It was suggested 
that soccer-specific play during childhood provided the necessary practice 
constraints for players to develop sustained improvements in anticipatory and 
decision making skills. 

A key issue that still remains unclear concerns identification of what it is 
about the specific activities being practiced that is responsible for the 
development or transfer of anticipatory skills. While the work of Berry et al. 
and Roca et al. both highlight the value of deliberate play, they differ in the 
proposed manner in which anticipatory skills may be developed. While Berry 
et al. highlight the transfer value of invasion activities, Roca et al. suggest 
sport-specific game play is key. Whichever model is considered, both lack 
detail concerning what are the underpinning component processes of these 
activities that are so beneficial for anticipatory skill development. For 
example, is it the similarity of the rules, relative density of player numbers, 
size of the playing field, similar visual scanning demands, or number of 
decision making choices, to name but a few possible factors? Prospective 
investigation of these component processes is required to better understand 


the specificity and transfer potential of such activities. 


Experimental Studies of Anticipatory Skill Learning 


The increasing awareness of the role of anticipation in the development of 
expertise has led many to question the best way to improve predictive 
performance. Specific programs of perceptual training have been designed to 
accelerate the speed at which learners acquire anticipatory skill. These 
programs most frequently require learners to make anticipatory judgments 
while watching occluded video clips that simulate real-life scenarios. Given 
the increasing understanding of the mechanisms that underpin the expert 
advantage in anticipation, early studies of perceptual learning largely sought 
to teach this information to learners in a highly explicit and prescriptive way. 
For instance Abernethy, Wood, and Parks (1999) used a series of training 
sessions to teach novice squash players the cues used by experts to anticipate 
the direction of shots played by opponents. A training group who received 
this information while observing temporally occluded video footage of 
opponents experienced a significant pre- to post-test improvement in 
anticipation that was not apparent for either a placebo or control group. Other 
studies that have used this prescriptive approach (e.g. Farrow, Chivers, 


Hardingham, & Sachse, 1998; Singer et al., 1994) have largely supported the 


claim that perceptual training can accelerate the rate at which anticipatory 
skills are acquired. 

The growing awareness of the benefits of skills that are learned in an 
implicit manner has led some to question the utility of the highly explicit 
prescriptive method of perceptual training and instead to trial more implicit 
approaches to learning. Studies from motor learning show that skills that are 


learned implicitly, that is, without the accrual of explicit rules for how to 


perform the task, are, when compared to explicitly learned skills, less likely 
to be forgotten and more robust under stress (Masters, 1992). This has led to 
perceptual training approaches that either reduce or eliminate the amount of 
guidance provided about how to perform the task. The guided discovery 


approach reduces the amount of guidance (e.g. Magill, 1998), generally by 


directing attention toward the regions of the display known to contain critical 
information but without providing any explicit rules to help interpret that 
information. This approach has been shown to result in a rate of learning that 
is at least as fast as that possible when using explicit learning (Abernethy, 
Schorer, Jackson, & Hagemann, 2012), but with the benefit of being more 
resistant to stress (Smeeton, Williams, Hodges, & Ward, 2005). In contrast to 
cognitive discovery learning, perceptual discovery learning appears to indeed 
rely more on implicit than explicit forms of processing (Raab et al., 2009). 

Another related approach has been the use of color cueing, where a 
semi-transparent colored patch is used to highlight to the learner the usual 
locations in which a skilled performer would direct their gaze (Hagemann, 
Strauss, & Canal-Bruland, 2006; Ryu, Kim, Abernethy, & Mann, 2012; 
Savelsbergh, Gastel, & Kampen, 2010). This approach, though, does not 
appear to be any better than what is possible when using regular guided 
discovery (Abernethy et al., 2012; Klostermann, Vater, Kredel, & Hossner, 
2015). 

Other implicit approaches generally use secondary tasks or incidental 
learning to minimize the explicit rules generated during learning. Abernethy 
et al. (2012) used incidental learning to encourage the implicit learning of 
anticipatory skill in handball goalkeepers (see also Farrow & Abemethy, 
2002). Instead of asking participants to anticipate the direction of an 


opponent’s throws seen in video footage, participants were asked to decide 


whether pairs of occluded clips were the same, before watching the 
unoccluded clips to verify if the correct response was made. Abernethy et al. 
argued that this approach encouraged anticipatory information to be learned 
incidentally because it required (unrelated) perceptual judgments to be made 
on the basis of advance information and, moreover, that by viewing the 
unoccluded footage, a link would be made between the occluded information 
and the subsequent ball flight. Consistent with studies of motor implicit 
learning, the implicit perceptual training resulted in a rate of learning that was 
marginally slower than what is found when using explicit or guided discovery 
learning, but participants benefited from improved retention of the 
anticipatory skill when tested five months after the intervention, 
demonstrating the long-term benefits of implicit perceptual training. 
Smeeton, Huys, and Jacobs (2013) used novel manipulations of the basic 
kinematic information contained within a tennis stroke to change the 
information content of either local or global body cues and they offered 
suggestions as to how the selective presentation of biological motion patterns 
might accelerate the acquisition of anticipation skill. 

Studies of perceptual training show that anticipatory skill can be 
improved by a variety of different approaches; however, further work is 
required to fully appreciate the usefulness of that training for improving on- 
field performance. A number of attempts have been made to evaluate changes 
in on-field anticipatory performance as a result of video-based training, with 
some studies finding successful transfer (Farrow & Abernethy, 2002; 
Hopwood, Mann, Farrow, & Nielsen, 2011; Scott, Scott, & Howe, 1998; 
Williams, Ward, & Chapman, 2003) and others not (Farrow et al., 199 


Singer et al., 1994). A possible concern for effective transfer is that video- 


a 


based training largely requires perceptual judgments to be made that may not 


fully replicate what takes place when movements are performed as would 
typically occur in the performance environment (see van der Kamp et al., 
2008). Approaches that maintain the relationship between perception and 
action may augment transfer, for instance if using on-field training or virtual 


reality (Vignais, Kulpa, Brault, Presse, & Bideau, 2015). 


Summary and Concluding Remarks 


Superior anticipation is one of the distinguishing characteristics of expert 
performance. The evidence for differences in anticipatory skill between 
experts and non-experts is well established — what requires increasing focus 
in the coming decades is not further verification of this attribute but rather a 
greater understanding of the basic neural mechanisms underpinning expert 
anticipation and enhanced understanding of how best to develop training 
programs and interventions to hasten the development of anticipatory skill. 
The former necessitates a fusion of behavioral measures of anticipation with 
the rapidly advancing tools of neuroscience and in this regard the growing 
interest in the brain mechanisms of expertise is encouraging (Yarrow, Brown, 
& Krakauer, 2009). The latter requires the development of a higher quality 
evidence base than presently exists. In contrast to the evidence that exists in 
other fields of human endeavor where the efficacy of interventions is 
rigorously assessed — biomedicine being a clear example — the absence of 
longitudinal data and data derived from double-blinded randomized control 
trials in the skills training literature is marked. Addressing this will be 
fundamental to seeing the growing knowledge about the nature of expert 
anticipation translated into approaches that might accelerate the acquisition of 
anticipatory skills by non-experts. As the examples in this chapter illustrate, 
sport provides an ideal domain in which to study skilled anticipation (e.g. 
Williams & Ericsson, 2005) and consequently much of what is known 


regarding skilled anticipation has been derived from studies in this domain. 


Greater study of anticipation in other domains of human performance in 


future is also highly desirable. 
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The word “expert” has the same word origin as “experience” and 
“experiment” and thus refers to someone who has learned from experience. 
Exactly how some individuals attain an exceptional level of performance 
while gaining their extended experience has not been _ explained. 
Consequently, experts have been viewed for centuries as mysterious and are 
sometimes revered, much like those considered to be geniuses. As recently as 
the early 1960s, Herbert Simon co-authored a paper (Simon & Simon, 1962) 
which argued that grand masters of chess at that time were typically viewed 
as “intellectual prodigies, who performed feats of memory and discovery 
unachievable by mere mortals” (p. 425). It was only with the emergence of 
programmable computers that scientists started to propose theories and 
explicit models for how experts were able to generate their superior 


performances. 


In their pioneering work, Newell and Simon (1972) proposed models of 
the cognitive processes of experts and how computer programs could 
simulate important aspects of their thought processes by explicitly describing 
how information was accessed, processed, and stored in different memory 
stores. They gave examples for how relatively simple computer programs can 
perform intellectually challenging tasks, such as solving complex integration 
problems and finding solutions to difficult puzzles. These demonstrations 
encouraged researchers to start studying the acquisition and structure of 
expert performance. Their pioneering research on expertise focused on 
describing the accumulated knowledge of experts and its storage in long-term 
memory (LTM) with its vast capacity and how this knowledge was activated 
in short-term memory (STM) with its very limited capacity to guide the 
selection of actions and behavior. 

Perhaps the most striking empirical phenomenon that distinguished 
experts and novices was the experts’ ability to reproduce complex stimuli 
from game situations after only a brief exposure to eliminate the possibility 
that the experts stored information in LTM. However, the main superiority of 
memory was restricted to representative situations from their domains of 
expertise and did not reflect general memory superiority. The original 
accounts of the experts’ superior working memory were based on a clear 
distinction between the transient storage in the severely capacity limited STM 
and the nearly permanent storage in the virtually unlimited LTM (Chase & 
Simon, 1973; Simon & Chase, 1973). Over time researchers started focusing 
on working memory and how experts were able to generate complex plans 
during activities, such as selecting moves in chess, representing complex 
situations during reading or playing team sports, executing complex 


transformations of input during simultaneous interpreting from one language 


to another, executing performances of complex piano pieces from memory, 
and calculating complex multiplication and addition problems mentally. 

This chapter will start with a historical background and then review how 
recent research has searched for general characteristics of the superior 


working memory of experts. 


A Historical Background 


The start of modern laboratory research on expertise is generally attributed to 
the research by Herbert Simon and Bill Chase (Simon & Chase, 1973). They 
proposed that skilled chess players retrieved moves by recognizing patterns 
of chess pieces (chunks) that had been previously stored in LTM. They 
explained expertise in chess as a gradual acquisition of increasingly complex 
patterns (particular configurations of chess pieces). 

Many decades earlier Djakow, Petrowski, and Rudik (1927) had found 
that skilled chess players had superior memory of chess positions, but not 
when other types of materials were presented in memory studies. Later 
Adrian de Groot (1978) examined how highly skilled chess players identified 
the best move for a position while thinking aloud. He found that with even 
very short presentations of a chess position from an unfamiliar game, world- 
class players were able virtually perfectly to reproduce all the pieces in the 
position. In contrast, less skilled club players were only able to recall the 
locations of a fraction of the presented pieces. 

In a classic series of studies, Chase and Simon (1973) presented chess 
positions briefly (5 seconds) to players of differing levels of chess expertise 
and requested immediate recall of each position by placing pieces on a chess 
board. They also presented chess boards with randomly arranged chess pieces 
under the same conditions followed by immediate recall. For positions from 
chess games, the chess master recalled many more pieces correctly than the 
two less skilled chess players. In contrast, recall of the randomly scrambled 


chess positions was equally poor regardless of skill and all three players 


recalled only a handful of chess pieces correctly. Chase and Simon (1973) 
argued that all chess players regardless of skill were able to hold about the 
same number of chunks (around seven) in STM as proposed by Miller 
(1956). Miller (1956) found recall to be between five and nine chunks (seven 
chunks plus/minus two) in laboratory experiments with brief sequential 
presentation of more traditional stimuli, such as digits, consonants, and 
unrelated words. The differences in the amount recalled by expert and less 
skilled chess players could be explained by the number of chess pieces 
belonging to each chunk, where experts had more complex chunks with 
several pieces and less skilled players might only have a single piece per 
chunk. This model may explain the superior selection of moves and greater 
recall performance for briefly presented chess positions as a function of 
larger and more refined chunks in LTM. 

Chase and Simon (1973) provided a new laboratory paradigm for 
studying differences in immediate memory between experts and less skilled 
individuals for representative stimuli from game situations and for 
random/unstructured stimuli. Their original finding which showed an expert 
advantage in memory recall for representative situations, but not for 
unstructured domains, has subsequently been replicated in many other 
domains, such as bridge (Charness, 1979; Engle & Bukstel, 1978), Go 
(Reitman, 1976), Othello (Wolff, Mitchell, & Frey, 1984), medicine 
(Norman, Brooks, & Allen, 1989), electronic circuit diagrams (Egan & 


Schwartz, 1979), computer programming (McKeithen, Reitman, Rueter, & 
Hirtle, 1981), dance (Starkes, Deakin, Lindley, & Crisp, 1987), basketball 
(Allard, Graham, & Paarsalu, 1980), field hockey (Starkes & Deakin, 1984), 
and volleyball (Bourgeaud & Abernethy, 1987) (for more complete reviews, 
see Ericsson & Lehmann, 1996; North, Williams, Ward, & Ericsson, 2011). 


Limits and Criticisms of the Simon—Chase Theory of Expertise 


The central mechanism of the Simon—Chase theory of expertise (Simon & 


Chase, 1973) involved the slow acquisition of more complex and refined 


patterns (chunks) in LTM and the association of these recognized patterns 
with appropriate chess moves, which could be retrieved efficiently when 
selecting moves for encountered chess positions. In their classic chapter, 
Chase and Simon (1973) discussed many aspects of chess players’ cognitive 
mechanisms and processes, such as their use of the mind’s eye for planning 
and their ability to encode sequences of chess moves in LTM. It was, 
however, their theory of expertise-based chunks to retrieve actions from 
capacity-limited STM that provided the generalizable hypotheses for 
predictions about experts’ superior memory. 

One of the initial criticisms concerned the assumption that storage of 
briefly presented chess positions was restricted to STM with its very limited 
capacity. For a dissertation supervised by Bill Chase, Neil Charness (1976) 
presented a chess position for 5 seconds and this was immediately followed 
by the player engaging in an unrelated task that demanded the full capacity of 
STM. After working on the unrelated task for some time the chess player was 
instructed to recall the chess position. If the chess position had been only 
stored in STM then there would be no information in STM to use for its 
recall. The observed recall disproved Chase and Simon’s (1973) assumption 
that storage in LTM was not possible with presentation times of 5 seconds — 
less than the 8 seconds minimal time for storage of chunks in LTM proposed 
by Simon (1974). Charness (1976) found that skilled players could recall 


most of the information even after the intervening task implying storage in 


LTM. If the information from a briefly presented chess position was stored in 
both LTM and STM then the capacity limits of STM would not constrain the 
amount of recall. 

Bill Chase was also involved in another research project that showed 
that performance on tasks, assumed by Miller (1956) and Newell and Simon 
(1972) to measure the fixed capacity of STM, could be dramatically 
improved by training. In an extended training study, Bill Chase and I (Chase 
& Ericsson, 1981, 1982; Ericsson, Chase, & Faloon, 1980) showed that 
memory performance could be dramatically improved on the digit-span task, 
where digits are presented at 1 s/digit with immediate recall. One participant 
increased his digit-span from a normal level of 7 digits to over 80 digits. A 
number of individuals have dramatically increased their memory span 
performance with training (Ericsson & Kintsch, 1995). With practice in 
remembering several chess positions each presented for 5 seconds and 
immediately followed by the next, the recall of multiple chess positions was 
dramatically increased (Gobet & Simon, 1996). A chess master was able to 
recall information from 5—10 chess positions while using LTM to store 
information in templates (Gobet & Simon, 1996). 

A more general criticism was directed toward one of the assumptions of 
the Simon—Chase theory of expertise, namely the assumption that the 
memory task involves the same cognitive mechanisms as representative tasks 
for measuring chess expertise, such as selecting the best move for a chess 
position. There is evidence that tournament performance in chess is more 
closely associated with performance on selecting-the-best-move task than 
with the memory performance for briefly presented chess positions (see 
Ericsson, Patel, & Kintsch, 2000, for a review). Furthermore, other 


researchers showed that mere practice with feedback could improve memory 


performance, even for chess positions. A participant without any prior chess 
experience was able to increase her recall level of briefly presented chess 
positions from that of a beginner to that of a chess master within 50 hours of 


training (Ericsson & Harris, 1990; for a published description, see Ericsson & 


Oliver, 1989). The effects of training on memory for briefly presented chess 


positions were later replicated by Gobet and Jackson (2002). These studies 
suggest that performance on the task of recalling chess positions can be 
improved dramatically in 10-50 hours, which implies that this performance 
can be mediated by factors that are not related to those determining 
performance in chess tournaments, which has been found to require years and 
decades of chess study (Simon & Chase, 1973). 

Another assumption by Chase and Simon (1973; Simon & Chase, 1973) 


has been questioned, namely that increased experience in the domain is 
always associated with superior memory performance for representative 
stimuli in the domain. As stated earlier, there is a correlation between amount 
of experience in a given domain of activity and performance on memory 
tasks with stimuli from the domain of expertise. There are, however, 
important exceptions. Ericsson and colleagues (Ericsson & Lehmann, 1996; 
Ericsson et al., 2000) enumerated several examples of highly experienced 
experts who failed to demonstrate superior memory. For example, expert 
musicians did not show superior memory for melodies that were auditorily 
presented compared to less skilled individuals (Sloboda & Parker, 1985), 
expert actors were not superior to less skilled individuals for memorizing text 


(Intons-Peterson & Smyth, 1987; Noice & Noice, 1997), and map experts did 


not show better memory for the traditional types of maps (Gilhooly, Wood, 
Kinnear, & Green, 1988). Experienced doctors recall less information after 


diagnosing presented patient vignettes than advanced medical students with 


less experience (Patel & Groen, 1991; Wimmers, Schmidt, Verkoeijen, & van 


de Wiel, 2005). The experienced doctors are more accurate in their diagnoses, 
thus showing how superior memory and superior diagnostic performance can 
be dissociated. 

These findings about superior memory performance cannot be easily 
explained by Chase and Simon’s theory based on increased complexity of 
chunks as function of extended experience of engaging in activities in the 
domain. The following review will be based on the expert performance 
approach to expertise, where the goal is to understand the structure and 
acquisition of mechanisms that mediate reproducibly superior performance in 
activities that capture the essence of expertise in the domain, such as winning 
chess games, superior performance in sports, typing, music, and simultaneous 
translation. Within this conceptual framework, the question becomes how 
superior performance is supported by expanded working memory. 
Furthermore, how are the mechanisms mediating this working memory 


acquired or do they reflect innate superior capacity? 


The Expert Performance Approach to 
Superior Working Memory 


When the goal is to account for experts’ superior performance, it is essential 
to study the performance that captures its essence in the domain of expertise 
(Ericsson & Smith, 1991; Ericsson & Ward, 2007; Ericsson & Williams, 


2007). The example of superior memory for briefly presented chess positions 


is not a defining characteristic of chess performance. Chess players do not 
practice this particular memory task. In fact, we showed in the previous 


section that specific practice on that memory task improves performance 


dramatically without increasing chess ability for non-chess players. The core 
idea of the expert performance approach is to identify cognitive activities that 
are tightly connected to the defining expert performance. In the case of chess, 
it seems almost inevitable that a player who increases his/her ability to select 
the best move for any chess position will also be more successful in playing 
chess games. In fact, de Groot (1978) has already demonstrated that superior 
chess performance can be captured by presenting chess positions and asking 
participants to select the best move. Subsequently, researchers have shown 
that this general method of presenting representative situations from real 
chess games and requiring participants to generate the best available actions 
provides the best measure of chess skill that predicts performance in chess 


tournaments (Ericsson et al., 2000; van der Maas & Wagenmakers, 2005). In 


the second step of the expert performance approach, the focus is to identify 
the mechanisms that mediate the reproducibly superior performance. This is 


the step where it is likely that superior working memory of experts might be 


one of the mechanisms that can account for how they are able to generate 
their superior actions in representative situations. In the third and final step, 
the focus is on finding the genetic, developmental, or practice factors that 
were responsible for the mechanisms mediating superior performance. With 
respect to superior working memory mediating superior performance, I 
discuss if and how domain-specific activities might provide practice to refine 
this mechanism or whether general working memory capacity might provide 
a necessary prerequisite for the expert performance. 

The approach of capturing performance has some important advantages. 
In order to measure performance reliably it is necessary to give participants 
similar tasks many times. When experts are given representative tasks from 
their domain of expertise, it would be very surprising if a few more hours of 
testing would change the performance of skilled participants with hundreds 
or thousands of hours of related performance (Ericsson, 1996; Ericsson & 
Smith, 1991). In contrast, when experimenters give participants (experts and 
novices) a task that they have never encountered, they are often able to 
improve their performance substantially after a few hours of practice, as 


demonstrated by the training studies reported earlier. 


Capturing Superior Performance on Memory 
Tasks 


In most types of expertise, the expert does not practice to improve their 
memory directly, but rather the superior working memory is a unintentional 
consequence of efforts to improve the selection and execution of superior 
actions, such as selecting the best chess move or generating the best action 
for a particular situation during a game or a professional activity like surgery. 
There are, however, memory experts, whose expertise is defined by superior 
performance on specific memory tasks. In the case of memory experts, they 
have explicitly practiced specific memory tasks in order to improve their 
performance. 

In the last decades there are memory competitions, such as the World 
Memory Championship (WMC). The WMC is a worldwide competition in 
memory activities (1992 — now). During an annual competition, participants 
compete against each other in ten events: Names and Faces, Binary Numbers, 
Hour Numbers, Abstract Images, Speed Numbers, Historic Dates, Hour 
Cards, Random Words, Spoken Numbers, and Speed Cards. These 
competitions can be viewed as the culmination of centuries of studies of 


individuals with superior memory performance (Ericsson, 1985; Wilding & 


Valentine, 2006). When individuals acquire skills to memorize random lists 


of digits or words, they report finding ways to encode the arbitrary 
information by meaningful associations and it is easier to study their 
performance with the expert performance approach and design experiments 


testing hypotheses about the mechanisms mediating superior performance. 


The structure of the mechanisms developed by memory experts has been 
hypothesized to mediate the domain-specific superior memories of experts 
according to theoretical frameworks for experts’ exceptional working 


memory (Ericsson & Kintsch, 1995; Ericsson et al., 2000). 


Mechanisms Mediating Superior Memory Performance for 
Numbers 
Exceptional memory performances have been observed and reproduced in the 
laboratory since Binet’s (1894) original studies of mental calculator, Inaudi, 
and the mnemonist, Diamondi. Some of the subsequent studies (for a review, 
see Ericsson, 1985) conducted more extensive interviews about the 
performers’ encoding methods. The relative contribution of training and 
innate memory capacities could not be assessed because all the research was 
conducted with individuals with an existing exceptional performance and 
thus information about its development was not available. Other researchers 
studied the effect of practice on particular memory tasks, but these studies did 
not produce performers with clearly exceptional memory performance. In an 
attempt to replicate one of these early training studies (Martin & Fernberger, 
1929), Bill Chase and I (Chase & Ericsson, 1981, 1982; Ericsson et al., 1980) 


provided training on the digit-span task for two participants and collected 


retrospective reports on their cognitive processes during a memory trial to 
assess any changes as a function of increases in performance. These two 
participants dramatically improved their digit-span performance and reached 
spans of around 80 digits and over 100 digits, respectively. More recently, 
Feng Wang recalled 300 digits perfectly (Memory and Mental Calculation 
World Records, 2015). 


A large number of laboratory studies have uncovered the mechanisms 


mediating these instances of exceptional memory performance and found 
them consistent with acquired skills based on rapid storage in and retrieval 


from LTM as proposed by Ericsson and Kintsch’s (1995) paper on long-term 


working memory (LTWM). The basic assumptions of this model are that 
individuals acquire memory skills to encode rapidly the presented 
information in LTM, where they attach associations to retrieval cues 
organized in retrieval structures. The retrieval cues kept in STM can then be 
used to retrieve the encoded presented information from LTM. For example, 
retrieval cues can be acquired to allow reproduction of the serial order of long 
sequences of digits, but they can also permit retrieval of information based on 
associated categorical content of the generated encodings as described by 
Ericsson and Kintsch (1995). LTWM corresponds to domain-specific and 
even task-specific memory skills that are acquired for storage of particular 
information that is efficiently retrieved in specific situations. LTWM is 
fundamentally different from other theoretical concepts of general working 
memory where information is held transiently in a store with fixed general 
capacity rather than a store in LTM expandable with the increased acquisition 
of task-specific skills. An account of experts’ performance and expanded 
working memory capacity will require an explanation of how the associated 
skills were developed and acquired by engagement in appropriate practice 
activities. The proposed skills were designed to provide an expanded working 
memory for information related to a particular task and therefore would need 
to be developed in tight coordination with the target performance. In the 
exceptional case of developing superior performance on a memory task, there 


would not be an additional task to adapt to. 


Chess 


Tournament chess performance can be captured by standardized tasks in a 
controlled environment. As described in an earlier section, experimenters 
have presented chess positions to players with the explicit task of finding the 
best next move. In his pioneering research, de Groot (1978) tried to identify 
the processes mediating superior performance by instructing his participants 
to think aloud while they generated their best move. De Groot (1978) found 
that the chess players first examined the chess position and then generated 
some promising moves. They then turned to planning and a more systematic 
evaluation of the consequences of making one of the possible first moves by 
examining a likely counter-move or moves by the other chess player and then 
generating moves countering those moves and so on. The think-aloud 
protocols show that skilled chess players evaluate the consequences of 
mentally moving some 5-15 chess pieces on the chess board, which would 
imply that the players must be able to maintain a large amount of information 
accessible from working memory. When de Groot (1978) examined the 
protocols for a chess position from both world-class chess players and skilled 
club players, he found to his surprise that their planned move sequences did 
not differ in their depth. Subsequent research with several different chess 
positions found that higher chess skill is associated with deeper search 
(Charness, 1981). 

The benefit of planning has been questioned by theories that propose 
that the best move options are directly retrieved intuitively from memory 


without the need for additional planning. In his pioneering research, de Groot 


(1978) found many instances where the world-class players discovered their 
final move while they were engaged in the systematic search and planning. 
Similarly, Saariluoma (1992) found that planning reduced the frequency of 
mistakes and failures to consider important aspects of the chess position. 
Moxley, Ericsson, Charness, and Krampe (2012) analyzed a number of think- 
aloud protocols from move selection tasks and identified the chess moves 
first mentioned and analyzed as well as the final move selected. The main 
finding was that chess players at all skill levels were able to improve upon 
their initially accessed chess move by engagement in planning with one 
exception — for easy move selection tasks expert chess players seem to access 
the best move as their first mentioned alternative. 

To gain an advantage from planning and search, players need to have 
generated an accurate memory representation of the chess positions generated 
at the end of a sequence of planned moves and counter-moves. Highly rated 
chess players are more able to accurately evaluate chess positions resulting 
from the execution of more moves than less skilled players, whose ability 
deteriorates quickly with each additional move generated (Holding, 1985; 
Holding & Pfau, 1985). Cowley and Byme (2004) found that at the end of 


long plans more skilled chess players are more accurate in their evaluations 


of the corresponding chess position than less skilled players. 

A more direct method of assessing skilled chess players’ working 
memory is to study chess play and move selection under blindfold conditions, 
where they cannot see a chess board, but have to rely on their memory to 


know where the chess pieces are located. Ericsson and Oliver (for a 


description, see Ericsson & Staszewski, 1989) studied a candidate chess 
master who lacked prior “blindfold” training yet was able to play at a very 


high level of chess without seeing the chess board. There is a sizable body of 


evidence that chess masters are all able to play blindfold (without a visible 
board) showing the current position, at a level close to their normal skill level 
(Karpov, 1995; Koltanowski, 1985). Chabris and Hearst (2003) did not find a 


significant difference between move quality from blindfold and normal chess 


games and Jeremic, Vukmirovic, and Radojicic (2010) found only a very 
small decrease in play quality during rapid chess (with limits on the time for 
selecting moves). 

More controlled studies have presented groups of chess players with 
long sequences of chess moves and asked them to update the chess position 
in the mind after each move. At the end of the sequence, the experimenter 
would test to see if the players’ mental representation accurately represented 
the correct chess position. These experiments show that chess masters, but 
not less skilled players, are able to mentally update multiple chess games 
when an experimenter reads the associated sequences of moves from multiple 
chess games (Saariluoma, 1991, 1995). One of the most intriguing studies 
was conducted by Saariluoma and Kalakoski (1998). They did not allow the 
chess masters to see the whole chess position at once, but presented it 
auditorily by having the experimenter speak the location of each of the 
individual pieces. Not only were the chess masters able to integrate this 
piecemeal information, but they were able to examine the position mentally 
and then select the best move. 

The evidence from studies of chess experts’ ability to generate memory 
representations to maintain efficient access to a lot of relevant information is 
highly consistent with the proposal that memory skills based on LTM have 
been acquired with the explicit goal of supporting working memory required 
for particular types of activities. These acquired skills that support planning 


can explain the superior memory for briefly presented chess positions. 


Vincente and Wang (1998) observed that no chess players spend time 
practicing their memory for briefly presented chess positions. For chess 
players “skilled memory is actually only a by-product of chess skill, rather 
than the other way around” (Vincente & Wang, 1998, p. 48). This issue raises 
the issue of what can explain the individual differences in acquired chess skill 
and the correlated skills related to their superior working memory to support 


some task activities. 


Practice Related to Improvements in Working Memory Support of 
Chess Playing 

The specificity of chess players’ superior memory implies that engagement in 
certain types of chess-related activities might account for its increase. The 
critical aspect of working memory in chess playing involves being able to 
mentally transform a chess position and then plan long sequences of moves 
while accurately representing the chess relations in a resulting chess position. 
The question is how one can get immediate feedback about incorrectly 
generated chess positions or falsely represented critical relations between 
chess pieces. Ericsson, Krampe, and Tesch-Rémer (1993) proposed that 
simulating playing against grand masters by studying records of each move in 
one of their games would provide an appropriate practice activity. The 
players could then compare their generated best moves to the moves selected 
by a grand master for that same position and receive immediate feedback 
about the correctness of their selected move. If the chess players’ selected 
move was inferior then the player would need to recall how they generated 
their move and try to figure out why the best move had not been considered 
or, at least, not been selected. 

It is likely that failures to find and select the best move for a given game 
position can be linked to failures to generate the consequences of a series of 
moves fully and accurately. Efforts to address these failures would involve 
either changes of the mental representations of the critical chess positions or 
deeper planning to identify unforeseen consequences, which would both 
likely lead to the improvements in their working memory representations. In 


support of this account, several researchers have found that the amount of 


accumulated time that a chess player has engaged in serious solitary study of 
chess is highly correlated with attained performance (Charness, Tuffiash, 
Krampe, Reingold, & Vasyukova, 2005). In contrast, merely playing chess 
games does not provide immediate feedback after each move and the 
accumulated amount of time playing chess has a lower correlation with chess 
skill (Charness et al., 2005; Ericsson & Moxley, 2012). 


Working Memory during Mental Calculation 


Calculation of mathematical problems involving addition, subtraction, and 
multiplication is relatively straightforward and is solved by generating 
intermediate products that can be written down on paper or by using other 
forms of external memory, such as an abacus. By removing access to any 
external memory aids, the task of calculating the result mentally is 
transformed into a challenging task due to the required storage of 
intermediate results that need to be kept accessible for future processing 
(working memory). To preserve the relevance to expert performance, the 
review will be restricted to individuals who exhibit reproducibly superior 
performance in mental calculation. 

Smith (1983) provides an extensive review of all the great mental 
calculators. Virtually without exception Smith (1983) found that these mental 
calculators relied on the traditional 9 x 9 multiplication table for single-digit 
combinations and thus would have to expand their working memory to 
maintain generated intermediate products. He also concluded that the superior 
memory for numbers of the calculators was not a cause of their performance, 
but rather a consequence of their acquired skill to perform mental 
calculations. He found no evidence for a superior general memory ability of 
these calculators (for a similar conclusion, see Ericsson, 1985). Subsequently, 
studies of world-record-breaking mental calculators found no evidence for 
any “unusual basic capacities” (Jensen, 1990, p. 259) and an analysis of a 
calculating prodigy found superior memory only for numbers (Pesenti, Seron, 
Samson, & Duroux, 1999). 


There are few efforts to trace the processing by “think-aloud” protocols 
and verbal reports. Chase and Ericsson (1982) found that an expert at 
squaring numbers had developed several encoding strategies for storing the 
presented numbers and some of the intermediate products in LTM with 
associated retrieval cues. In a study involving several hundred hours of 
training with mental multiplication, Staszewski (1988) showed dramatic 
improvements in speed. Part of the improvement was related to the trainee’s 
generation of an internal representation of the numbers in LTM, which 
provided faster access and processing. Evidence for storage in LTM with 
retrieval cues was supported by the trainee’s ability to recall the majority of 
the numbers of the presented multiplication problems at the very end of the 
testing session. 

Most of the research demonstrating expanded working memory during 
addition of numbers has been conducted on highly skilled individuals trained 
to use an abacus, which is a manual device for calculation developed in 
China many centuries ago. With specialized training, some of these 
individuals are able to conduct the calculations without access to a physical 
abacus and they report relying on a mental representation of an abacus. 
Several investigators have measured the memory capacity and representation 
in individuals differing in the level of skill in calculation with a mental 
abacus. When measured with the digit-span test, highly expert individuals 
have an exceptional memory performance (Hatano, Amaiwa, & Shimizu, 
1987), but somewhat less skilled individuals performed in normal range 
(Hatano, Miyake, & Binks, 1977). The highest abacus performers have 
exceptional digit-spans between 12 and 16 digits, but their memory span for 
other types of material, such as letters and fruit names, is indistinguishable 


from the normal range of performance (Hatano & Osawa, 1983). The 


development of a mental abacus is not an automatic consequence of extensive 
calculation with an actual abacus; mental abacus calculation requires practice 


specifically on mental addition problems (Stigler, 1984). Developing a 


complex mental abacus is a slow process that starts with the ability to 
represent the sum consisting of three or four digits. Expanding the mental 
abacus by one additional digit is estimated to take about one year of 
deliberate effort (Hatano & Osawa, 1983). 


Working Memory during Simultaneous 
Translation 


One of the domains of expertise with an obvious demand for extensive 
working memory is simultaneous translation. During simultaneous 
interpreting, the interpreter has to be both listening to the speaker in 
Language A and at the same time speaking the previously spoken message in 
Language B. The translation in Language B lags the current speech generated 


in Language A by two to six words (Christoffels & de Groot, 2005). 


Numerous investigators have conducted research measuring the general 
working memory capacity of professional interpreters, bilinguals, and 
monolinguals. Researchers have not found consistent significant differences 
on tests for general working memory capacity between interpreters compared 
to regular students and students in interpreting schools (Képke & 


Nespoulous, 2006; Liu, 2008). In a comprehensive review, Signorelli (2008) 


showed that influences of age, level of interpreting experience, and 
differences in testing method confounded outcomes and seemed to explain 
the observed pattern of differences or lack of differences with two exceptions. 
Professional interpreters perform consistently better on tests of non-word 
repetition and score higher on the reading span test. Signorelli (2008) argued 
that these types of superiority reflect acquired ability developed in response 
to the professional demands of interpreters. Interpreters need the ability to 
store non-words in memory to be able to maintain names of people referred 


to in the interpreted message. Interpreters also need to hold on to the meaning 


of presented items while comprehending the next sentence, which is similar 
to the task of measuring reading span. 

Ericsson (2010) considered research on expert performance in 
interpreting, where several interpreters perform the same set of interpreting 
tasks and are scored for accuracy. In one of a small number of studies, 
Dillinger (1994) found that professional interpreters translated significantly 
more units of meaning than a group of untrained bilinguals. In a longitudinal 
study, Gerver, Longley, Long, and Lambert (1984) examined changes in 
performance during an intensive course for interpreters and found that the 
best predictor of the quality of simultaneous translation at the end of the 
course was their ability to generate synonyms and missing words in texts. 
More recent studies show that expert interpreters excel in tasks involving 
elaborative encoding and storage in LTM (Kopke & Signorelli, 2012) and 
displayed superior listening skill as demonstrated by superior recall of a prose 
passage (Hiltunen & Vik, 2017). These findings suggest that professional 
interpreters do not reach an automatic phase, but keep monitoring the 
translation process by developing increasingly complex representations 
(Ericsson, 2010). Consistent with this hypothesis, Moser-Mercer (2000) 
documented how interpreters improve their performance by gradually 
eliminating certain types of mistakes. More recent articles propose that their 
findings on expert interpreters are better explained by the interpreters’ 
acquisition of complex task-specific representations and skills rather than 
individual differences in general basic memory capacities (Hiltunen, 
Paakkonen, Vik, & Krause, 2016; Hiltunen & Vik, 2017; Strobach, Becker, 
Schubert, & Kiihn, 2015). 


Sports and Working Memory 


Many types of sports do not seem to be overly taxed by demands on memory 
for the current situations while generating one’s performance. In some 
competitive sports, the performance needs to be generated in response to the 
changing external demands (open sports), such as volleyball, tennis, and 
soccer, whereas other sports are referred to as closed sports, where the 
performer should be in control of their performance in domains such as ice- 
skating, gymnastics, and diving. Allard and Burnett (1985) suggested that 
open sports would be more likely to reveal cognitive demands than the closed 
sports. Some of the earliest research demonstrated that basketball players 
demonstrated superior immediate recall of game situations to non-players, 
but no superiority for unstructured stimuli (Allard, Graham, & Paarsalu, 
1980). In a subsequent study, Starkes (1987) presented slides showing game 
situations in field hockey as well as unstructured situations. National level 
players recalled the locations of more players (around 45 percent) than 
varsity players (around 35 percent) in game situations but no differences for 
unstructured situations (around 20 percent). In spite of this significant 
expertise advantage in recall, this study of slides with many players (up to 22 
players) demonstrates a clear selectivity of recall. Subsequent studies have 


also been able to demonstrate superior memory associated with more 


expertise (Ward & Williams, 2003), but the recall performance has not been 
directly connected to superior representative performance during matches. 
In team sports, it is possible to identify game situations where a given 


player has the ball and needs to make a quick decision about passing, 


shooting, or dribbling. Helsen and colleagues (Helsen & Pauwels, 1993; 
Helsen & Starkes, 1999) presented slides of game situations in soccer and the 
soccer players were asked to state their recommended action and found 
superior accuracy in selecting actions and faster reaction times for semi- 
professional players over kinesiology students. They found a similar 
expertise advantage when they projected video sequences of game situations 
on the wall and had players actually execute their selected action by kicking a 
ball. In order to select the correct action, players need to be able to represent 
mentally where their teammates and opposing players are as well as where 
they are heading. They need to be able to rapidly update this information as 
players are constantly moving. Of particular interest to issues of working 
memory, is the information that the players cannot see and thus have stored 
in working memory. Research on players’ eye fixations within the visual 
field on various cues is discussed in the chapter on anticipation (Abernethy, 
Mann, & Farrow, Chapter 35, this volume). Many researchers let the picture 
of the game situation or the last image of a video sequence remain visible 
while the participants report on their considered alternative actions, which 
does not force the players to rely on their memory of the final game situation 
(Johnson & Raab, 2003; Raab & Johnson, 2007; Ward, Ericsson, & 
Williams, 2013, Exp. 1; Ward & Williams, 2003). In contrast, Ward et al. 
(2013) replaced the final game situation with a blank screen, thus forcing the 
participant to rely on memory when reporting on options for action. They 
found that both skilled and less skilled soccer players generated two to three 
options and the main difference was that skilled players more often generated 
the best options (as assessed by coaches) as the first option and also 
generated much fewer poor options than the less skilled players. How the 


skilled soccer players generate their superior representations of the current 


game situation has primarily been inferred from differences in their eye 
fixations. Research on eye fixations and head movements during actual 
soccer games has found that skilled players who are able to complete 


successfully more passes engage in more active visual search prior to 


receiving the ball (Spearritt, 2013). This is consistent with the hypothesis that 
building a more accurate representation of the relevant players’ positions in 
the concurrent game situation requires both more frequent head movements 
and more eye movements. The working memory representation of game 
situations needs to be easy to update without interference from memories of 
previous game situations. 

According to theories based on individual differences in general 
working memory capacity, there should be a correlation between working 
memory capacity and accuracy of skilled handball players’ selection of 
actions. Furley and Memmert (2015) failed to find a significant correlation 
between these two variables and concluded that “working memory is not a 
limiting factor in creative decision making amongst skilled performers” (p. 
1). 


Working Memory in Other Domains of 
Expertise 


Working memory has been extensively studied in other domains, such as 
typing, reading music (sight-reading), and reading text aloud. These domains 
are particularly attractive for researchers because the participants provide a 
virtually continuous stream of observable behavior, which can be scored for 
accuracy and speed. Superior performance is often closely associated with a 
longer span between the executed actions, such as pressed piano keys or 
typing keys, and the presented information fixated with the eyes at the exact 
same time. Skilled individuals look further ahead in the text or music score 
than less skilled individuals in sight-reading (Sloboda, 1974; Thompson, 
1987), typing (Salthouse, 1984), and reading aloud (Gibson & Levin, 1975). 


Research on skilled typing has not focused on the relation between 


individual differences in working memory and typing performance, although 
it is generally accepted that eye—hand span is the best predictor of typing 
speed. In his classic study, Salthouse (1984) found no significant relation 
between typing speed and traditional measures of short-term memory, such as 
the memory span. 

Musicians and accompanists who can perform a piece of music with 
limited or no preparation have developed the skill of playing music by sight 
(sight-reading) and they can often easily outperform concert pianists at 
playing an unfamiliar music piece directly from the music score without any 


practice (Lehmann & Ericsson, 1993). A few researchers have measured 


participants’ performance on tests for general working memory capacity and 


sight-reading performance. In the first study, Thompson (1987) measured 
working memory by a memory span test in flute players at different levels of 
music skill. This study did not find any significant relation between memory 
span and music skill. In subsequent studies Kopiez and Lee (2006, 2008) 
administered a psychometric battery, which included measures of complex 
working memory. In one study, student musicians and accompanists were 
interviewed about their current and past practice as alternative predictors of 
individual differences in sight-reading skill. In their most comprehensive 
analysis of their data, Kopiez and Lee (2008) found that speed of alternating 
finger movements and amount of accumulated sight-reading experience were 
the only significant predictors of sight-reading performance, but not working 
memory. In an earlier analysis of the relation between level of sight-reading 
skill and working memory, Kopiez and Lee (2006) examined performance as 
a function of the difficulty level of the sight-reading task. For the three lowest 
difficulty levels (Levels 1-3) Kopiez and Lee (2006) found a significant 
correlation between working memory and sight-reading performance. For the 
second highest difficulty level (Level 4), the correlation with working 
memory was no longer significant and for the highest difficulty level (Level 
5) the correlation with working memory was not significant (r = 0.08). 

In a more recent study, Meinz and Hambrick (2010) both collected 
psychometric measures of working memory and estimated the accumulated 
number of hours of past and current music practice along with sight-reading 
performance. They found that even after controlling for the amount of piano 
practice individual differences in the measures of working memory made a 
small but significant additional prediction of sight-reading performance (r = 
0.28). Meinz and Hambrick interpreted these findings to demonstrate the 


influence of stable basic working memory differences on any level of 


acquired sight-reading performance. An alternative account of their findings 
proposes that Meinz and Hambrick (2010) included a sample of individuals 
with vastly different sight-reading ability. For example, some participants had 
never engaged in sight-reading and others had only played the piano for one 
year. Extrapolating the findings of Kopiez and Lee (2006) the inclusion of 
data on lower levels of sight-reading would lead to a correlation between 
working memory and sight-reading performance. Hence, all the findings from 
these studies are consistent with Ericsson and Chamess’s (1994) hypothesis 
that expert performers acquire associated working memory skills and LTWM 
mechanisms and thus do not need to rely on nor do they rely on basic general 
capacities measured by tests of complex span. In a recent review Ericsson 
(2014) showed that correlation between cognitive ability and performance of 
beginners in several domains is often significant. Consistent with Ericsson 
and Charness’s (1994) hypothesis of the acquisition of new cognitive 
mechanisms in more skilled performers the correlation between cognitive 
abilities and performance among skilled and expert performance disappeared 
and was no longer significant. In a recent meta-analysis on the relation 
between cognitive abilities, in particular fluid intelligence, and chess 
performance, Burgoyne et al. (2016) found that beginners and less skilled 
chess players revealed significant correlations, but these correlations 
decreased with increased skill and were no longer significant for samples of 


chess masters. 


Summary and Conclusion 


This review documents how reproducibly superior performance in a wide 
range of domains depends on expanded access to intermediate products, and 
refined mental representation of the current situation, which allow the experts 
to generate actions that are more appropriate than those of less accomplished 
performers. The experts’ working memory has been highly adapted to the 
special demands of executing the expert performance in a particular domain 
and is thus closely adapted to the structure of tasks and the essence of 
expertise in that domain. 

Since Chase and Simon’s (1973) pioneering research quantifying the 
superior memory for chess positions of chess experts, the focus has been 
redirected toward understanding how experts’ memory encodings permit the 
processing involved in recognizing features of chess positions and planning 
and evaluating superior moves. Simply showing that experts display superior 
performance on memory tasks compared to less skilled individuals for 
presentations of stimuli from the domain does not necessarily help us 
understand how the superior memory performance is related to the superior 
expert performance, such as winning chess tournaments. Contemporary 
research shows that an expert level of performance on a task measuring 
memory for briefly presented stimuli from the domain of expertise can be 
attained by a total novice after 50-100 hours of practice (less than 5 percent 
of the time taken to attain the expert performance in the domain, such as 
reaching a master level in chess). More accumulated knowledge about a 


domain increases memory for presented information paired with items from 


that domain, even when that paired information is completely fictitious and 
not directly related to the domain (Bellezza & Buck, 1988; Van Overschelde 
& Healy, 2001). For example, an individual with more knowledge about 
baseball would more likely be able to recall the beverage in a presented 
sentence than someone with less knowledge, when that sentence described a 
famous baseball player drinking a particular type of beverage (Van 
Overschelde & Healy, 2001). The difference between overall memory and 
selective memory of relevant information was demonstrated for medical 
students and doctors. After diagnosing a patient vignette an experienced 
medical doctor cannot recall as much detailed information as an advanced 
medical student (Patel & Groen, 1991; Wimmers et al., 2005). The doctors’ 


superior reasoning ability reflects higher-level representations that they have 


acquired to support reasoning about clinical alternative diagnoses (Ericsson 
& Kintsch, 1995; Ericsson et al., 2000; Patel, Arocha, & Kaufmann, 1994). 


The higher-level representations do not contain all of the extraneous details 


and therefore the doctors will recall less information than the medical 
students, who haven’t been able so far to acquire these representations. 
Expanded working memory for relevant information to support superior task 
performance reflects refinement of encodings in working memory of experts. 

These findings are consistent with a more general pattern of findings 
arguing that expert performance is special and different from many types of 
performance studied in the laboratory, where participants only have one or a 
small number of hours of experience with the task. A recent review 
(Ericsson, 2014) found that the influence of general abilities, such as IQ, is 
greater on performance of beginners but virtually disappears for individual 
differences among expert performers. The general argument is that increased 


levels of practice and training lead to the acquisition of mechanisms that 


mediate the improved performance. At higher levels of performance the 
acquired mechanisms appear to mediate most of the individual differences in 
expert performance. This view is controversial and two recent meta-analyses 
(Macnamara, Hambrick, & Oswald, 2014; Macnamara, Moreau, & 
Hambrick, 2016) have found that the reported accumulated practice does not 
account for all or even most of the variance, but recent commentaries have 
pointed out the differences between different types of practice and their 
differential impact on performance improvement (Ericsson, 2014, 2016; and 
see the more complete discussion of these issues by Ericsson, Chapter 38, 
this volume). 

These general results have important implications for the study of the 
structure and acquisition of expert performance. Research on the structure of 
expert performance requires that researchers can capture the conditions for 
which the performance has been shaped and refined. Unless the tasks and 
conditions provide the ecologically valid constraints for the target 
performance, the observed performance will be different and thus may not 
activate the mechanisms mediating the genuine expert performance 
(Ericsson, Chapter 38, this volume). Similarly, any research on the acquired 
working memory mechanisms needs to reproduce the expert performance in 
order to measure and describe how these mechanisms can meet the particular 
needs for accessible storage of information. 

Longitudinal studies will be necessary to describe the detailed 
development of the structure of the expert performance and how mechanisms 
mediating working memory storage are first established and then developed 
and refined to meet the changing memory demands of the developing 
performance. A related research question could focus on how various types 


of practice activities can effectively modify the mechanisms mediating the 


performance. Studies that can monitor the training as well as assess the 
structure of the performance before and after the training would permit us to 
describe the extended process of improving expert performance as well as 


potential aspects that cannot be completely changed by any type of training. 
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Introduction 


In thinking about expertise, we often focus on skilled physical performance 
(e.g. the world-class tennis player or gymnast) or skilled decision making 
(e.g. the chess grand master). In addition to these aspects of performance, 
however, situation awareness (SA), an up-to-date understanding of the state 
of the world and systems being operated, forms a critical cornerstone for 
expertise in many complex domains, including driving, aviation, military 
operations, and medical practice. The characteristics that allow people to 
develop high levels of SA often develop silently alongside more observable 
features like skilled physical performance, even in tasks such as sports that 
are considered primarily physical in nature. 

Take, for example, the following excerpt from a magazine story about 
Wayne Gretzky, an all-time leading hockey scorer who set or tied 49 
different National Hockey League records including most goals, most points, 


and most assists. 


Gretzky doesn’t look like a hockey player ... His shot is only average — 
or, nowadays, below average ... Gretzky’s gift, his genius even, is for 
seeing ... To most fans, and sometimes even to the players on the ice, 
hockey frequently looks like chaos: sticks flailing, bodies falling, the 
puck ricocheting just out of reach. But amid the mayhem, Gretzky can 
discern the game’s underlying pattern and flow, and anticipate what’s 
going to happen faster and in more detail than anyone else in the 
building. Several times during a game you’ll see him making what seem 


to be aimless circles on the other side of the rink from the traffic, and 


then, as if answering a signal, he’ dart ahead to a spot where, an instant 
later, the puck turns up. 


(McGrath, 1997) 


Although undoubtedly Wayne Gretzky possesses the important physical 
skills associated with the sport of hockey, this article points out that the 
critical attribute that places him head and shoulders above his contemporaries 
is mental — his ability to understand what is happening in the game and to 
anticipate where the puck will be. This superior SA allows him to be “ahead 
of the game” and outmatch bigger, faster, and physically better players. 

Similar stories can be found in other sports, for example football, 
basketball, or tennis, in which anticipating the actions of one’s teammates, 
one’s opponents, and where the ball is going are all key to effective 
individual and team performance. The importance of SA can even be found 
in a relatively straightforward sport such as golf which, at first glance, might 
appear to be only a matter of a mechanical match between the golf swing and 
the distance and bearing to the hole. However, even in this sport SA has a 
role. Expert players will walk the course ahead of time to observe key 
situational features that make one course play differently from another. 

In the 2004 Masters Golf Tournament, Phil Mickelson won by sinking a 
putt on the last hole. Credited as critical to that action was the fact that the 
player immediately prior to Mickelson putted from almost the exact same 
location. This allowed Mickelson to “read” the hole — observing the winds, 
very slight variations of the grass, and grades of the slope between the ball 
and the hole. He consciously worked to develop the best SA possible before 


taking his swing. 


SA plays an equally important role in other domains, such as military 
operations, healthcare, piloting, or air traffic control, where there are many 
factors to keep track of which can change quickly and interact in complex 
ways. For example, pilots must keep track of other aircraft, terrain, system 
status, and warning lights along with their relevant characteristics, 
conformance with clearances, distance to terrain and airports, time to landing, 
distance available on fuel remaining, and the seriousness of any system 
malfunctions for the safety of flight, as well as projections of changes in 
weather (Endsley, 2009). An anesthesiologist must keep up with changing 
patient vital signs and appearance, actions of the surgeon, diagnoses of 
patient state, and projected changes in patient state as effected by potential 
actions and interventions (Schulz, Endsley, Kochs, Gelb, & Wagner, 2013; 
Schulz et al., 2016; Wright, Taekman, & Endsley, 2004). Effective decision 


making depends on high levels of SA and thus so does effective performance. 


This chapter will discuss the ways in which SA is critical to expert 
performance and the factors that allow it to improve with the development of 
expertise in a domain. Studies from several different arenas will be presented 
to highlight the many difficulties that novices have in developing good SA 


and to show how SA improves as individuals develop expertise. 


Situation Awareness 


A general definition which has been found to be applicable across a wide 
variety of domains describes SA as “the perception of the elements in the 
environment within a volume of time and space, the comprehension of their 
meaning and the projection of their status in the near future” (Endsley, 1988, 
p. 97). This definition of SA can be further discussed in terms of the three 


levels of SA embodied within it. 


Level 1 SA: Perception 


The perception of relevant information from the environment forms the first 
level of SA. Without basic perception of important information (through 
visual, auditory, tactile, or other means), the odds of forming a correct picture 
of the situation decrease dramatically. In highly complex and demanding 
environments, novices may have significant difficulty in knowing which 
information is most important or in accessing needed information in a timely 
manner to form Level 1 SA. They may also have a lower understanding of 
the many subtle cues that are relevant to more experienced decision-makers 
and may therefore fail to look for them. 

Even with considerable expertise, Level 1 SA can be quite challenging 
in many complex and dynamic domains. Jones and Endsley (1996) for 
example, found that 76 percent of SA errors in pilots could be traced to 
problems in perception of needed information (due to either failures or 
shortcomings in the system or cognitive processes). A similar study in 
hospital-based critical care found 46.6 percent of SA errors were at Level 1 
(Schulz et al., 2016). 


Level 2 SA: Comprehension 


SA involves more than simple perception of information — it also demands 
that people understand the meaning and significance of what they have 
perceived (Level 2 SA). Thus it encompasses how people combine, interpret, 
store, and retain information, integrating multiple pieces of information to 
arrive at a determination of their relevance to the person’s goals. This is 
analogous to having a high level of reading comprehension, as compared to 
just reading words. Twenty percent of SA errors in pilots have been found to 


involve problems with Level 2 SA (Jones & Endsley, 1996). In hospital- 


based critical care, 38.7 percent of SA errors involved problems with 
developing a correct understanding of the information available (Schulz et al., 
2016). 


Level 3 SA: Projection 


At the highest level of SA, the ability to forecast future situation events and 
dynamics (Level 3 SA) marks individuals who have the highest level of 
understanding of the situation. This ability to project from current events and 
dynamics to anticipate future events (and their implications) allows for timely 
decision making. Experts rely heavily on future projections as a hallmark of 


skilled performance. While only 3.4 percent of aviation errors (Jones & 


Endsley, 1996) and 12 percent of hospital-based critical care errors (Schulz et 
al., 2016) were associated with failures to make accurate projections, this 
primarily reflects the significant problems that people have in achieving even 


Levels 1 and 2 SA in these environments. 


SA Model 


A general cognitive model of SA will be presented, followed by a discussion 
of how expertise affects critical processes involved in developing SA in 
complex domains. The model of SA in Figure 37.1 shows how SA feeds into 
decision making and performance in an ongoing cycle (Endsley, 1995, 2004, 
2015a, 2015b). 
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Figure 37.1 Model of situation awareness in dynamic decision making 
(Endsley, 1995). 
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The model describes many mechanisms and cognitive features of the 
individual that determine whether a person will develop good SA, given the 


same environment and equipment as others. These features include: 


1. Perceptual processing. A number of factors influence the perceptual 
process, including (a) the salience of cues that may be registered pre- 


attentively (Neisser, 1967; Treisman & Paterson, 1984) in a bottom-up 


processing manner, (b) an understanding of relative priorities and 
frequency of information change from long-term memory (Wickens, 
1992), and (c) active goals and information already perceived (Braune & 
Trollip, 1982) which create goal-driven processing. As novices often 
lack this knowledge, they tend to have much more scattered information 
search patterns (Mann, Williams, Ward, & Janelle, 2007; Stein, 1992; 
Yu, Wang, Li, & Braithwaite, 2014). Experienced pilots seem to have 
more automated skill in extracting information, and a more refined 
mental model of cross-coupling and leading indicators to guide that 
search (Bellenkes, Wickens, & Kramer, 1997). 


2. Limited attention. Since human capacity for attention is limited, the 
ability of the individual to apply conscious focalized attention to all 
relevant information is severely constrained in most domains. For 
example, Kuipers and colleagues (Kuipers, Kappers, van Holten, van 
Bergen, & Oosterveld, 1990) found that lack of attention to primary 
flight instruments (56 percent) and too much attention to target planes 
during combat (28 percent) were major causes of aircraft accidents in 
combat. Too much attention to some aspects of the situation negatively 
affects awareness of others. Information from different modalities may 


be more readily processed in parallel (Wickens, 1992), somewhat 


reducing this effect; however, the limits of attention for taking in needed 


information form a significant challenge for SA. 


3. Limited working memory. Much of SA requires working memory 
for storing, integrating, and processing perceived information and 
maintaining the current internalized model of what is happening. As 


working memory is constrained, critical information may be forgotten or 


may not be properly integrated to develop Level 2 or 3 SA. Working 
memory is a bottleneck only for novices and those in novel situations, 
however, as in practice long-term memory structures can preclude such 
limitations. Sohn and Doane (2004) for example, found that expert pilots 
rely more on long-term memory for SA than on working memory, and 
only novice pilots were working memory constrained. Gonzalez and 
Wimisberg (2007) showed that Level 1 SA improved over time with 
experience on a task, and its relationship to working memory decreased 
accordingly. Endsley (1990, 2000) also found that SA data are available 
even five or six minutes after a scenario freeze supporting a model of 
cognition in which working memory is an activated subset of long-term 


memory (Cowan, 1988). 


4. Goal-driven processing alternating with data-driven processing. 
Goals can be thought of as ideal states of the system or environment that 
the individual wishes to achieve. In a top-down, goal-driven process, the 
person’s goals and plans will direct which aspects of the environment 
are attended to in the development of SA (Casson, 1983). This creates 
significant efficiencies in information processing when the individual 
has correctly prioritized goals and criticality. Conversely, in a bottom- 
up, data-driven process, the individual’s attention is directed across all 
relevant information (perhaps in a learned scan pattern) and will be 
captured by salient features or by key information which can indicate to 
the individual that a different class of situation is present for which 
different plans or goals should be activated. An ongoing cycling 
between goal-driven and data-driven processing is a key feature 


underlying SA. “SA is largely affected by a person’s goals and 


expectations which will influence how attention is directed, how 
information is perceived, and how it is interpreted. This top-down 
processing will operate in tandem with bottom-up processing in which 
salient cues will activate appropriate goals and models” (Endsley, 1995, 
p. 49; see also Corbetta & Schulman, 2002). 


5. Prior knowledge and expectations. Information perception is 
directly affected by the contents of both working memory and long-term 
memory. Expectations play an important role in SA, affecting where 
people will look for information and how they interpret what they 
perceive. A person’s preconceptions or expectations about information 
affect the speed and accuracy of the perception of that information, with 
processing occurring more rapidly if the information is consistent with 
expectations (Jones, 1977), as well as advanced knowledge of the 
characteristics, form, and location of information (Barber & Folkard, 
1972; Davis, Kramer, & Graham, 1983; Palmer, 1975; Posner, Nissen, 
& Ogden, 1978). 


6. Rapid categorization of information. Long-term memory is used to 
rapidly classify perceived information into known categories or mental 
representations as an almost immediate act in the perception process 


(Hinsley, Hayes, & Simon, 1977). This categorization is based on 


integrated information, and tends to occur very rapidly (Ashby & Gott, 

1988). Highly detailed classifications (e.g. the specific model of aircraft 
or specific type of tree) are possible with higher levels of expertise, due 
to superior memory stores and a more well-developed understanding of 
the critical cues in the environment that allow experts to make very fine 


classifications. 


7. Mental models and schema. Mental models and schema provide 
cognitive mechanisms for interpreting and projecting events in complex 
domains. These long-term memory structures can be used to 
significantly circumvent the limitations of working memory. “When an 
individual has a well-developed mental model of the behavior of 
particular systems or domains, the model will provide (a) for the 
dynamic direction of attention to critical cues, (b) expectations regarding 
future states of the environment (including what to expect as well as 
what not to expect) based on the projection mechanisms of the model, 
and (c) a direct, single-step link between recognized situation 


classifications and typical actions” (Endsley, 1995, p. 44). 


It is worth noting that although the three levels of SA represent 
increasingly better SA, they do not necessarily indicate fixed, linear stages. In 
a linear progression, a person needs to see or hear a piece of information 
(Level 1 SA), interpret what it means (Level 2 SA), and then project what 
will happen next (Level 3 SA). This is from a data-driven perspective. In 
reality, however, a simple Level 1—2-3 progression is not a sufficiently 
efficient processing mechanism in a complex and dynamic system, which is 
where expertise and goal-driven processing come into play. 

The model recognizes that many times people are goal-driven. Based on 
their goals and their current understanding or projections (Levels 2 and 3 
SA), individuals will look for data to either confirm or deny their assessments 
or will search for missing information (look for Level 1 data). This is an 
iterative process, with understanding driving the search for new data and new 
data combining to build understanding, as represented by the feedback arrow 


in the model in Figure 37.1. “That is they use their level 2/3 SA to generate 


assumptions regarding level 1 representations (either rightly or wrongly). In 
this way people can have level 2 and 3 SA, even when they do not have 
complete or accurate level 1 SA, and can use the higher levels of SA to drive 
the search for and acquisition of level 1 SA” (Endsley, 2004, p. 318). For 


example, if a pilot sees that an engine overheat light is on, he might assess 


that there is a particular type of malfunction which will cause him to search 
for further information on displays to confirm or deny this assessment, and to 
determine the status of other systems that might affect or be affected by the 
engine. 

Inherent in any discussion of SA is also the notion of what is important. 
For a given individual, SA requirements are based on the goals and decisions 
related to their job or role. The air traffic controller does not need to know 
everything (e.g. the co-pilot’s shoe size and spouse’s name), but does need to 
know a great deal of information related to the goal of safely separating the 
aircraft within his or her air sector. Physicians have just as great a need for 
SA; however, the things they need to know about will be quite different, 
dependent on a different set of goals and decisions. Because the things that 
people need to perceive, comprehend, and project are by nature domain- 
specific, high levels of SA, and thus expertise, in one domain will not 
necessarily translate into high levels of SA or expertise in another domain. 

In addition, the model highlights key features of the environment and 
the systems available to support decision making that affect how well people 


are able to obtain and maintain SA, including: 


1. The capability of the systems provided for obtaining the needed 
information (e.g. relevant sensors, data transmission capabilities, 


networking). 


2. The design of the system interface that determines which information 
is available to the individual, along with the effectiveness of the format 


of the displays for transmitting information (Endsley & Jones, 2012). 


3. System complexity, including the number of components, 
interrelatedness of those components, and the rate of change of 
information, all affecting the ability of the individual to keep up with 
needed information and to understand and project future events (Endsley 
& Jones, 2012). 


4. The level of automation present in the system, affecting the ability of 
the individual to stay “in-the-loop,” aware of what is happening and 
understanding what the system is doing (Endsley, 2017; Endsley & 
Kiris, 1995). 


5. Stress, fatigue, and workload that occur as a function of the task 
environment, the system interface, and the operational domain, each of 
which can act to decrease SA. Specifically stressors (including anxiety, 
importance of consequences, noise, heat/cold, fatigue, mental load, and 
time pressure among others) can lead to (a) attentional narrowing, a 
decrease in attention to peripheral sources of information (Broadbent, 


1971), (b) premature closure, arriving at a decision without exploring all 


information available (Janis, 1982; Keinan, 1987), (c) poorly organized 


scan patterns(Janis, 1982; Keinan, 1987) which can negatively impact 


Level 1 SA, and (d) decrements in working memory capacity and 


retrieval (Hockey, 1986) which can additionally affect the higher levels 
of SA. 


Role of Expertise in Situation Awareness 


Based on this model of SA (Endsley, 1995), expertise in a particular domain 


has a significant role in allowing people to develop and maintain SA in the 
face of high volumes of information transfer and system complexity. These 
key factors first will be described, and then research supporting this model 
will be presented. The role of expertise on SA can best be explained in terms 


of two divergent ends of a continuous spectrum (Figure 37.2). 
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Figure 37.2 Factors affecting SA in novices and experts in a domain. 


Novices 


In the most extreme case, a person who is completely new to the systems and 
situations in a particular domain (e.g. a person learning to drive or fly for the 
first time) will be considerably overloaded in gathering information, 
understanding what it means, and formulating correct responses. Novices will 
be severely hampered in their efforts by both limited attention and working 
memory. Lacking other mechanisms, they will have to think through each 
piece of data and try to process it in working memory along with other pieces 
of data. Reading each gage, or listening to audio input, and interpreting those 
data will impose a significant burden in even mildly complex systems. 
Understanding the significance of what is perceived will likely be error-prone 
as well, as a novice will not have the experience base from which to interpret 
perceived data. So a new driver, pilot, or power plant operator simply 
remains far behind the demand curve in taking in and processing the 
information that forms the basis of good SA in these dynamic environments. 
The process is further compromised by inefficiency. Novices lack 
knowledge of which information is most important at any given time. Scan 
patterns tend to be sporadic and non-optimal. They may neglect certain key 
information or over-sample other information unnecessarily. The problem is 
not just a matter of needing to learn a set way of taking in information. 
Without knowledge of the underlying relationships among system 
components, novices do not realize what information to seek out following 
receipt of other information or how to effectively prioritize. Seeing rising 


temperatures, for example, does not lead to the immediate shifting of 


attention to check the system pressure to determine whether an explosion is 
imminent. 

Thus, the prototypical novice is quickly overloaded, inefficient, and 
error-prone in developing SA. Decision making and performance are highly 
compromised as a result. Luckily, the development of expertise in a particular 
domain significantly reduces these problems through a number of 


mechanisms. 


Experts 


The SA model (Endsley, 1995) also details how with increasing experience in 


a particular domain, people are able to develop a number of mechanisms that 
help to overcome these significant hurdles. These include the use of mental 
models, schema, and automaticity, as well as the acquisition of a number of 


skills for efficiently gathering information and achieving SA. 


Mental Models 


Expertise in many arenas is highly dependent on mental models of the 
systems being operated and the operational domain. For example, a pilot 
develops not only a mental model of how the aircraft operates, including its 
many subsystems and its aerodynamic performance in the physical 
environment, but also a mental model of flight operations including air traffic 
control (ATC) procedures and expected behaviors associated with interacting 
with ATC and with other pilots. 

Mental models have been succinctly defined as “mechanisms whereby 
humans are able to generate descriptions of system purpose and form, 
explanations of system functioning and observed system states, and 


predictions of future states” (Rouse & Morris, 1985, p. 7). They are generally 


used to describe a person’s representation of some physical system (e.g. how 
an engine or computer works), but also can concern other types of systems 
(e.g. how a university or business works). 

Mental models embody stored long-term knowledge about these systems 
that can be called upon to direct problem-solving and interaction with the 


relevant system when needed. They may even be borrowed to shed light on 


similar systems. Although mental models grow and evolve with experience, 
they largely represent relatively static knowledge about the system — its 
significant features, how it functions, how different components affect others, 
and how its components will behave when confronted with various factors 
and influences — as opposed to the more situationally transient knowledge 
that is a part of SA. 

Mental models are highly useful in the process of developing SA 
(Endsley, 1988, 1995). A well-developed mental model provides several 


advantages: 


1. The mental model embodies knowledge regarding which aspects of 
the system or events are relevant in a given situation. This knowledge is 
critical for directing attention while taking in and classifying 
information in the perception process, making that process much more 
efficient, particularly in situations where there is a large amount of 
information to be processed. Experienced pilots are better at directing 
their attention to critical cues, for example (Schriver, Morrow, Wickens, 
& Talleur, 2008). 


2. The mental model provides a means for integrating various elements 
to form an understanding of their meaning (Level 2 SA). Understanding 
the significance of perceived system information is often very difficult 
without a mental model. Is a temperature of 104 degrees good, bad, or 
indifferent? Does this temperature mean the patient is in a critical or 
non-critical state ? Understanding is often not just a matter of 
interpreting one piece of data, but rather is a function of the integration 
of multiple pieces of data. For example, a physician will often consider 


multiple factors about the patient to determine a diagnosis. Mental 


models provide the basis for interpreting perceived information 
(singularly or together) in terms of the individual’s goals to form Level 
2 SA. 


3. The mental model is a mechanism for projecting future states of the 
system and environment (Level 3 SA) based on its current state and an 
understanding of its dynamics. The projection of future system states is 
very difficult without a mental model. If the temperature of a car is in 
the red zone, what is likely to happen? An experienced mechanic or 
driver has the knowledge base to project a cracked engine block, 
whereas an inexperienced person would not. In some cases, these 
projections could result from fairly simple pieces of knowledge (such as 
rules), but in many cases would require a far more detailed 
understanding of the nature of the system’s components and their 
interactions with each other. Accurately projecting what the enemy in a 
battle is likely to do requires a very detailed mental model not only of 
the battlefield (terrain features, weather, obstacles), but also of the 
enemy (objectives, capabilities, doctrine, culture, tactics, techniques, and 


procedures). 


4. Mental models also allow experts to make decisions even when faced 
with incomplete data, because mental models can provide “default” 
values to substitute for missing information via Q-morphisms (Holland, 


Holyoak, Nisbett, & Thagard, 1986). For example, an experienced pilot 


who knows only the time of day and where he is located is able to 
correctly state not only where a passing aircraft overhead is going, but 
its airline, type of aircraft, approximate altitude, and the type of engines 


on that plane. This knowledge comes not from superior vision or 


hearing, but from detailed knowledge of the air routes and flight 
schedules for the local airport. Mental models provide a great deal of 
related information that can be inferred from only a few cues, allowing 
experienced decision-makers to have a great deal of SA on the basis of 


only limited data. 


Schema 


Over time, people will encounter many situations and from these will develop 
a set of prototypical situations, or schema, in memory. Schema can be 
thought of as prototypical states of the mental model (i.e. patterns consisting 
of the state of each of the relevant elements for various situation classes). By 
pattern-matching between the current situation and schema in memory, 
people can instantly recognize known classes of situations. Oh, this is just 
like what happened last month. These prototypical situations can be learned 
through direct experience, formal training, or vicariously via the case studies 
and storytelling that are endemic in many professions. This is just like what 
happened to the aircrew in the Azores accident. 

Pattern-matching to learned schema provides a considerable shortcut for 
SA. Rather than processing data to determine Level 2 or 3 SA (requiring 
working memory or exercising the mental model), that information is already 
a part of the schema and must merely be recalled. There is considerable 
evidence for this type of pattern-matching to learned schema or exemplars in 
many domains (Dreyfus, 1981; Endsley, 2004; Klein, Calderwood, & 
Clinton-Cirocco, 1986; Medin, Altom, & Murphy, 1984; Norman, Eva, 
Brooks, & Hamstra, 2006; North, Ward, Ericsson, & Williams, 2011). This 


pattern-matching provides not only a recognition of a particular type of event 


or situation, but also the meaning or significance of that situation type and 
expected ways that the situation will progress over time. Thus schema and 
mental models provide a significant shortcut for determining Levels 2 and 3 
SA. Where scripts are tied to these schema, decision making is even further 
simplified. 

A critical feature of schema is that new situations need not be exactly 
like previously encountered situations to achieve a match. Only a few critical 
cues may be required to lead to a match, or a near match. This is a result of 
categorization mapping, a best fit between the characteristics of the situation 
and the characteristics of known categories or prototypes (Casson, 1983; 
Chase & Simon, 1973; Dreyfus, 1981; Mayer, 1983). 

A significant hallmark of expertise in SA is not only the development of 
a large repertoire of prototypical schema, but also sensitivity to often subtle 
cues that differentiate when each applies, and when they don’t quite match to 
current conditions necessitating a fallback to the mental model to best 
determine comprehension and projection. For example, Jones and colleagues 
(Jones, Quoetone, Ferree, Magsig, & Bunting, 2003) found that experienced 
weather forecasters with the best performance in predicting the severity of 
threat of impending conditions were able not only to pattern-match, but also 
to use mental simulation (via the mental model), as needed, to arrive at their 
assessment. More poorly performing weather forecasters over-applied learned 
patterns and were not as attuned to their differences or limits in fitting current 
situations. In addition, it appears that some people are better at pattern- 
matching than others, and this attribute has been shown to be significantly 
correlated with SA (Endsley & Bolstad, 1994). 


Automaticity 


A third relevant characteristic of expertise is the development of 
automaticity. Automaticity is normally considered in terms of physical tasks 
(e.g. riding a bicycle, or steering the car while shifting gears and operating 
the clutch and fuel pedal simultaneously in a standard automobile). In these 
cases, as the physical actions are performed more automatically, less 
conscious attention and effort is required of the individual. Tasks that initially 
completely absorb the attention resources of the individual eventually are 
performed with little if any conscious thought at all. The decrease in demand 
on mental resources associated with automaticity of physical tasks provides a 
boon for SA, leaving more attention and working memory for attending to 
information and interpreting it. Freed from the need to concentrate on the 
demands of steering and shifting, the driver can give more attention to 
detecting potential traffic hazards, for example. 

Automaticity may also be considered in relation to more cognitive tasks. 
With a fixed pairing of stimulus to response, even more complex cognitive 
behaviors can develop to a level of automaticity (e.g. braking following the 
detection of tail lights or a red traffic light in front of the driver, and pressing 
the gas pedal when the traffic light turns green). A highly experienced driver 
may begin to process these cues and convert even cognitive portions of 
driving to a level of automaticity with little conscious awareness or attention, 
freeing up the mind for other important matters such as conversation with a 
passenger, thinking about what to have for dinner, or daydreaming. Such a 
State can be quite common for experienced drivers operating in very familiar 


environments (e.g. the drive from home to work each day). 


As SA requires “awareness” of information by definition, SA in such a 
situation may be fairly low, even though performance may be adequate (i.e. 
no wrecks are occurring). A reasonable question might be, so is SA really 
necessary in this case? I believe the answer is yes. The reason is that good 
performance involves not just the known “normal” situations, but also the 
many abnormal situations that can and do occur. People operating at a level 
of cognitive automaticity are not as attentive to cues that are outside the 
learned “routine.” Thus, if there is a new stop sign on that well-learned route 
home, many people will run right through the intersection without stopping 
because they are not alert to this important situational element (e.g. “looked 
but failed to see” error; Najm, Mironer, Koziol, Wang, & Knipling, 1995; 
Sabey & Staughton, 1975). If, on a particular day, they intend to stop at the 


store on their way home from work, they may drive right past the store, as 
seeing the store sign fails to trigger the significance of its presence. 

The low SA associated with cognitive automaticity is likely to 
negatively affect performance when the situation falls outside the bounds of 
the learned routine. For this reason, experts in various domains take extra 
steps to guard against the deleterious effects of cognitive automaticity. Pilots, 
for example, run procedural checklists to make sure they check each item 
they are supposed to and do not lapse into automaticity in checking critical 


information. 


SA Skills 


Finally, with experience in a domain, people learn many specific skills that 
are relevant to efficiently and effectively obtaining critical information. Pilots 


learn to communicate with ATC and to scan their instruments to make sure 


their knowledge of the situation does not get out of date. Air traffic 
controllers learn how to scan their radar maps efficiently based on traffic 
patterns in the sector. Military officers learn how to gather and disseminate 
key information on the radio, how to listen for information that is relevant to 
them while ignoring other information, and where to post troops at listening 
and observation posts to ensure that key information is gathered and passed 
on in the first place. 

Much of the gathering of SA is not just passive, waiting for key 
information to be presented, but is an active process. The information that is 
available to military pilots, for example, is dependent on how they set up and 
operate their radar (search patterns, where it is focused, modes), what 
frequency they tune their radios to, and when they request information from 
others. Thus, their actions determine what information they will obtain. 

The ability to communicate effectively in teams has also been found to 
be critical for gathering and sharing SA in many domains, in that much 
information can be distributed across team members who may or may not be 
collocated (Bolstad et al., 2007; Endsley & Jones, 2001; Endsley & 
Robertson, 1996; Gorman, Cooke, Pederson, Connor, & DeJoode, 2005; Xiao 
& Moss, 2001). 

Several other skills have been found to be consistent with high levels of 
SA among experts, including an ongoing active effort to project likely and/or 
high consequence events (e.g. pilots anticipating engine failures, commanders 
preparing for various types of enemy attacks, or physicians looking for 
negative reactions by patients to medications (Amalberti & Deblon, 1992; 
Endsley, 1995; Endsley et al., 2000)). These projections are used to create 
contingency plans for avoiding or dealing with negative events quickly 


should they occur. Thus experts appear to solve difficult problems almost 


effortlessly, primarily because they have already preloaded a plan for that 
eventuality. 

Coupled with active ongoing contingency planning, in many domains 
(e.g. aviation military operations, and sports) experts will do extensive pre- 
mission planning, creating a thorough understanding of the anticipated 
conditions and likely problems to be encountered. This serves to anchor their 
SA, significantly reduce cognitive load during the mission, and guide their 
search for information to match to their expectations over time. On the 
negative side, however, these plans can actually foster anchoring and 
confirmation bias, leading people to miss important cues when things are not 
as anticipated. It has been found that pre-mission planning needs to 
encourage decision-makers to anticipate many different possibilities 
regarding what might happen, and to actively contingency plan rather than 
building a strong set of expectations that may be wrong (Taylor, Endsley, & 
Henderson, 1996). 

Experts also have good meta-strategies, such as actively cross-checking 
the validity of their own SA, against either additional information, or others 
(Taylor et al., 1996). This process helps to avoid SA errors, or missing 
information that others hold, but may not have shared. 

In that interruptions, distractions (task-related as well as unrelated), and 
overall workload all create a significant challenge for SA, good task 
Management strategies are also very important. Schutte and Trujillo (1996), 
for example, found that the best-performing flight crews were those whose 
task management strategies were based on the perceived severity of tasks and 
situations, rather than simple event/interrupt driven or procedural strategies. 
The ability to accurately assess the importance and severity of events (a part 


of Level 2 SA) is critical to the successful execution of this task management 


strategy. All of these are examples of learned skills, tailored to each domain, 


that improve with expertise and contribute to higher SA among experts. 


Running the Gamut 


It is worth noting that the mechanisms underlying good SA in experts are 
very domain-specific. They may promote good SA in the learned domain 
(e.g. brain surgery), but will not likely transfer to good SA in another domain 
(e.g. aircraft piloting), although there could be some transfer between similar 
domains (e.g. military operations and emergency response). And even within 
a domain, when novel situations are encountered, these mechanisms are of 
more limited value. Relevant schema will not be present and the mental 
model may be incomplete for the novel situation at hand. The risk for the 
expert in such a situation may be over-pattern-matching (i.e. trying to use an 
inappropriate schema), or stretching the mental model beyond its limits to 
accommodate the new situation. 

Therefore, the SA of even people who are generally considered to have a 
high level of expertise in a domain may be hampered by the same factors in 
very novel situations that hamper novices. The experienced pilot who has 
never encountered or been trained for a particular anomaly will be challenged 
to process information in working memory to determine what is happening. 
She also may be inefficient in searching for relevant information to solve the 
problem, in much the same way as when she was a novice pilot (although it is 
likely that she will not be as badly off as a complete novice). 

Most people do not operate at the level of novice all the time or expert 
all the time, but rather move around in between, using combinations of these 
cognitive mechanisms depending on the situation at hand and the availability 


of key constructs (e.g. mental models and schema). 


Cognitive Abilities Underlying Expertise in SA 


Numerous researchers have sought to examine the cognitive differences 
underlying the SA abilities of experts. In studying experienced military 
fighter pilots, Endsley and Bolstad (1994) found a tenfold difference in their 
SA. That is, the SA of the highest performing pilot was ten times that of the 
lowest performing pilot on an objective measure of SA, and these differences 
were highly reliable with test-retest measures ranging from 0.99 to 0.92. 
These differences were not explainable by simple measures of flight hours or 
years of experience. They found that the pilots with better SA were 
significantly better at attention sharing, pattern-matching, spatial abilities, 
and perceptual speed. Similarly, O’Hare (1997) showed that the best glider 
pilots performed better on a divided attention task, whereas Gugerty and Tirre 
(1997) reported that people with better SA on a driving task performed better 
on measures of working memory, visual processing, temporal processing, and 
time-sharing ability. 

Thus attention sharing (or multitasking) appears to be an important 
underlying skill for SA in aviation and driving, and likely in a number of 
other complex domains where there are competing tasks and information. 
Pattern-matching skills also come into play, which makes sense given the 
role of schema in forming situation understanding. Evidence for differences 
in spatial ability and perceptual skills, such as perceptual speed, visual 
processing, and temporal processing also appears to be relevant to SA in 
some domains (Endsley & Bolstad, 1994). 

A number of researchers have examined the role of working memory in 


SA. While Gugerty and Tirre (1997) found a correlation between SA and 


working memory on a driving task, Sohn and Doane (2004) found that expert 
pilots rely more on long-term memory for SA than on working memory, and 
that only novice pilots were working memory constrained. Sulistayawati, 
Wickens, and Chui (2011) showed that experienced pilots with lower 
working memory spans performed as well as those with better working 
memories on SA queries. Endsley and Bolstad (1994) also found that 
working memory ability did not predict SA scores in experienced pilots. 

Gonzalez and Wimisberg (2007) found that while Level 1 SA improved 
over time with experience on a task, its relationship to working memory 
decreased at the same time. Gutzwiller and Clegg (2013) examined the role 
of working memory in SA, using a simulated task with student participants, 
and found no relationship between working memory and Level 1 SA, but a 
positive relationship with Level 3 SA. This research shows that the SA of 
experts is not severely constrained by working memory (nor uniquely 
contained in it), but rather is able to draw on long-term memory as well 
(Endsley, 2015b), significantly compensating for the limits of working 
memory. 

Other researchers have found evidence that experts develop better 
allocation of attention and sensitivity to critical cues. In a study of 
commercial airline pilots, Orasanu and Fischer (1997) found that better pilots 
paid more attention to cues signaling deteriorating weather and actively 
sought out weather updates which allowed them to plan in advance for 
missed approaches. They were much more attuned to the constraints imposed 
by system failures and reviewed other alternatives in light of those 
constraints. Better pilots gathered more information allowing them to make a 
better decision, whereas poorer performing crews went right to evaluating 


options. Researchers have also found that poorer performing pilots tend to 


interpret cues inappropriately and to both underestimate the risk associated 
with a problem and overestimate their ability to handle dangerous situations 
(Orasanu, Dismukes, & Fischer, 1993; Wiggens, Connan, & Morris, 1995). 


In summary, the development of mental models, schema, automaticity, 


and information management skills provide the mechanisms that allow 
people to transform from novice to expert in a domain. These mechanisms act 
to (a) reduce working memory limitations; (b) provide effective and efficient 
methods for information gathering, filtering, and integration; (c) allow 
effective decision making in the face of uncertainty and limited information; 
(d) create sensitivity to subtle cues that might otherwise be overlooked; and 
(e) allow for projection and anticipatory thinking which are key to 
contingency planning and proactive decision making. The development of 
effective skills for managing information acquisition, interruptions, 
competing tasks, communications, attention sharing, contingency planning, 
and cross-checking are also useful hallmarks of expert SA. In order to better 
understand the skills and mechanisms that are fundamental to expert SA, I 
will review a few case studies that explore how SA varies as a function of 


expertise. 


SA and Expertise in Aviation Pilots 


Much research on SA has focused on aviation, primarily in the realm of the 
military or commercial aviation pilot. These pilots are typically very carefully 
selected through a number of screening tests and highly trained. The training 
is rigorous — anyone not meeting the bar gets “washed out” early on — and is 
ongoing, with most pilots being required to demonstrate prowess on a variety 
of maneuvers and emergency tasks on an annual or semi-annual basis. As 
such, most SA research has concentrated on a population that is well along 
the continuum toward the expertise end of the scale. 

In addition to benefiting from the additional cognitive structures 
discussed previously, anecdotal evidence indicates that experienced military 
pilots who are very good at SA engage in several practices or skills that are 
beneficial (Endsley, 1989, 1995). First they “think ahead of the aircraft” 
during the flight. They are continually engaged in projecting what might 
happen, allowing them to be ready for events if they materialize. Amalberti 
and Deblon (1992) found that these pilots report they spend much of their 
discretionary time engaged in projection and “what-if” type thinking. They 
also engage in extensive pre-mission planning and briefings that form a basis 
for what they expect to encounter during the mission, likely based on 
cognitive representations that support planning and prediction (Ericsson & 
Kintsch, 1995). These expectations can lead to faster, more accurate SA when 
they are correct, but also can lead to significant SA errors if they are wrong 
(Jones & Endsley, 1996). 


In a study comparing less experienced general aviation (GA) pilots 
(mean experience level = 720 hours) to more experienced airline pilots (mean 
experience level = 6,036 hours), and line check airmen, considered to be 
among the best commercial pilots (mean experience level = 12,370 hours), 
Prince and Salas (1998) noted two key differences between the groups. First 
the amount of preflight preparation increased between the groups, with more 
experienced pilots focusing on planning and preparation specific to the flight 
and gathering as much information as possible about the conditions and flight 
elements. 

Second, there was more focus on understanding and projection at higher 
levels of expertise. GA pilots described themselves as passive recipients of 
information with an emphasis on information in the immediate environment 
(Level 1 SA). Line pilots dealt more at the level of comprehension (Level 2 
SA) and emphasized their active role in seeking out information. Check 
airmen were more likely than the other groups to focus on Level 3 SA, 
seeking to be proactive. They dealt with large numbers of details and the 
complex relationships between factors in this process. 

GA pilots are typically far less experienced and have less training than 
commercial and military pilots. These much less experienced and less current 
pilots account for the vast majority of aviation accidents and fatalities; 94 
percent of all US civil aviation accidents and 92 percent of US civil aviation 
fatalities involve GA pilots (NTSB, 1998). This accident rate has remained 
fairly stable for nearly two decades (AOPA, 1997). A 1989 NTSB review of 
361 GA accidents concluded that 97 percent of the probable causes were due 
to pilot error (NTSB, 1989). Trollip and Jensen (1991) attribute these pilot 


error accidents to the following factors, in order of frequency: 


e loss of directional control; 

e poor judgment; 

e airspeed not maintained; 

e poor preflight planning and decision making; 
e clearance not maintained; 

e inadvertent stalls; 

e poor crosswind handling; 

e poor inflight planning and decision making. 


Most of these issues indicate problems with pilot SA. 

Bolstad and colleagues (2002) conducted a more in-depth analysis of SA 
problems in low-time GA pilots. They examined 222 incident reports at a 
popular flight school that contained reported problems with SA. These 
reports were stratified into four different experience levels: (1) students 
working on a private pilot’s license, which typically requires 40 to 60 flight 
hours; (2) students working on an instrument pilot qualification whose 
experience levels are typically 75 hours and more; (3) students working on a 
commercial pilot license whose experience levels are usually 200 hours and 
more; and (4) students working on a multiengine rating whose experience 
levels are typically 200 hours and more. Overall, the least experienced group 
— those working on their private pilot’s license — experienced the greatest 
proportion of incidents involving SA problems, and the more experienced 
groups encountered the least. The total number of incidents per flight hour 


was Statistically significantly different between each student group. 


As shown in Figure 37.3, SA errors were distributed differently across 
the three levels of SA. Significantly more Level 1 SA errors were found in 
the lower experience level groups and fewer in the most experienced group. 
None of the four groups was significantly more or less likely to experience 
Level 2 SA errors, but there were significantly fewer Level 3 SA errors in the 
multiengine student group and significantly more in the other three groups. 
The lower overall SA error rate observed for the most experienced group was 


due to a decrease in Level 1 and Level 3 SA errors, but not Level 2 SA errors. 
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Figure 37.3 SA error causal factors across pilot groups. 


Overall, a number of problems were noted as particularly difficult, 
leading to the SA problems found across this group of relatively 


inexperienced GA pilots. 


1. Distractions and high workload. Many of the SA errors could be 


linked to problems with managing task distractions and task saturation. 


This may reflect the high workload associated with tasks that are not 
learned to high levels of automaticity, problems with multitasking, or 
insufficiently developed task management strategies. These less 
experienced pilot groups had significant problems in dealing with 


distractions and high workload. 


2. Vigilance and monitoring deficiencies. Although associated with 
overload in about half of the cases, in many incidents vigilance and 
monitoring deficiencies were noted without these overload problems. 
This may reflect insufficiently learned scan patterns, attentional 


narrowing, or an inability to prioritize information. 


3. Insufficiently developed mental models. Many errors in both 
understanding perceived information and projecting future dynamics 
could be linked to insufficiently developed mental models. In particular, 
the pilots had significant difficulties with operations in new 
geographical areas, including recognizing landmarks and matching them 
to maps, and understanding new procedures for flight, landings, and 
departures in unfamiliar airspace. They also had significant difficulties 
with understanding the implications of many environmental factors on 
aircraft dynamics and behaviors. Pilots at these relatively low levels of 
experience exhibited problems with judging relative motion and rates of 
change in other air traffic which affected their understanding of the 
environment. (Experts have been found to be more sensitive to motion 
in understanding dynamic scenes in other research as well; Williams, 
North, & Hope, 2012.) 


4. Overreliance on mental models. Reverting to habitual patterns 


(learned mental models) when new behaviors were called for was also a 


problem for this group. They failed to understand the limits of learned 


models and how to properly extend these models to new situations. 


In a further study, Bolstad and colleagues (Bolstad, Endsley, Howell, & 
Costello, 2002) conducted challenging simulated flight scenarios with both 
inexperienced and experienced GA pilots. The inexperienced group included 
ten private pilots with a mean GA flight experience level of 109 hours (range 
41 to 300). The experienced group included ten flight instructors with a mean 
GA flight experience level of 1,790 hours (range 300 to 9,000). Experienced 
pilot observers completed a rating form during each flight, indicating whether 
key expected behaviors were never, sometimes, or always performed, and 
rating each pilot’s SA and performance on a 1 to 7 scale, based on their SA 
ratings. 

In general, the experienced group outperformed the inexperienced 
group; however, closer examination showed that the groups were not 
internally consistent. Therefore, for purposes of analysis, they were 


redistributed into four groups: 


1. Novice pilots with low SA (scores of 1, 2, or 3 on the overall SA 


scale). 

2. Novice pilots with moderate SA (scores of 4 or 5). 

3. Experienced pilots with moderate SA (scores of 4 or 5). 
4. Experienced pilots with high SA (scores of 6 or 7). 


The mean SA scores of these four groups are shown in Figure 37.4 with 
the high scoring experienced pilots clearly outscoring the other groups across 
all levels of SA. The pilots who scored better within the novice and 


experienced groups were not necessarily those with the most flight hours. In 


the novice category two of the more experienced GA pilots (180 and 300 
hours) were rated as among the seven having low SA, and of the three novice 
pilots with moderate SA, two were fairly low-time pilots (55 and 80 flight 
hours). In the experienced category, while all of the very high-time pilots 
(more than 1,000 hours) were rated as having high SA, so were two pilots 
with only 450 hours. Of the four experienced pilots that were rated as having 
only moderate SA, one had 730 flight hours of experience. Flight hours were 


therefore not significantly predictive of group. 
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Figure 37.4 SA ratings for pilots across the four groups. 


In examining the key behaviors observed, those pilots who were scored 
as having better SA (in both the novice and experienced categories) all 
received much higher ratings for aircraft handling/psychomotor skills, 
cockpit task management, cockpit task prioritization, and ATC 
communication/coordination than did the pilots who were rated as having 
lower SA. A step-wise regression model, accounting for 91.7 percent of the 
variance in SA score across all pilots, included aircraft handling/psychomotor 


skill and ATC communication and coordination. 


While aircraft handling might be considered by some to be a primarily 
manual or psychomotor task and not one significantly involved in a cognitive 
construct like SA, this finding is in agreement with previous studies which 
have found a relationship between psychomotor skills and SA, presumably 
because of issues associated with limited attention (Endsley & Bolstad, 1994; 
O’Hare, 1997). The development of higher automaticity on physical control 
tasks helps free up the attention resources needed for SA. The observers 
noted significant problems with basic flight skills in the novice pilots and 
indicated they were quickly overloaded by the tasks. 

Keeping up with ATC communications (which are very rapid and 
specialized) was also found to be challenging for many of the novice pilots. 
They had to ask for many repeats of clearances and it used up their limited 
attention resources. This issue was much less of a problem for the 
experienced pilots. 

So, while it might be easy to consider the development of physical skills 
associated with task performance as separate from cognitive skills like SA, 
these studies indicate that developing the required physical skills to 
automaticity in order to offload attentional demands may be an important 
prerequisite for developing high levels of SA. Expertise in SA may not be 
possible as long as an individual must concentrate on the performance of the 
physical tasks involved. 

Among the experienced pilots with high SA, good aircraft handling 
skills and good task prioritization were noted frequently. Their performance 
was not perfect, but this group seemed more able to detect and recover from 
their own errors compared to other groups. Many were noted as flying first 
and only responding to ATC clearances or equipment malfunctions when 


they had the plane under control. 


The experienced pilots who were rated as having only moderate SA 
were more likely to have been noted as having difficulty in controlling the 
simulated aircraft and exhibited poorer prioritization and planning skills. 
Thus, in addition to physical performance (i.e. aircraft handling), skills 
associated with task prioritization were shown to be important for high levels 
of SA. As many environments where SA is important involve multitasking 
among competing goals, demands, and tasks, skills for multitasking, 
planning, and prioritization are important (Endsley & Bolstad, 1994). These 
skills appear to develop or be enabled through increasing levels of expertise 
in the domain. 

Overall, the Bolstad et al. study illuminated the considerable workload 
problems that inexperienced pilots face. Basic flight control and ATC 
communications quickly overloaded them and left little attention available for 
maintaining SA. Attentional narrowing and fixation added to this problem, 
which is an issue that can be worse under stress (Allsop & Gray, 2014; 


Bacon, 1974; Baddeley, 1972; Eysenck, 1982), particularly for novice pilots 


who are more constrained by working memory. Among GA pilots with more 
experience, task prioritization and task management skills were also 
important skills associated with the pilots rated as having better SA. This 
group had the additional cognitive resources (or knowledge bases) to think 


ahead of the aircraft and plan for contingencies. 


SA and Expertise in Army Infantry Officers 


In a very different domain from general aviation, SA is important for army 
operations. The battlefield commander must have a clear and up-to-date 
assessment of the enemy, his own troops, and the battlefield environment. SA 
provides the foundation for military decision making, and the framework in 
which all plans and actions are conceived. 

Rather than obtaining SA from a largely engineered world like the pilot 
does, the battlefield commander is directly embedded in a natural world, with 
fewer direct indications of needed information. The intentions and activities 
of others may be very difficult to discern. One’s own troops are generally 
widely dispersed. Noise, heat and cold, fatigue, poor weather, smoke, and 
rugged terrain are all common challenges. Enemy forces intentionally 
practice denial of information, misinformation, and deception that must be 
detected and interpreted correctly. Decision making in the face of uncertain, 
missing, and conflicting information is common. 

To gather SA, army officers have traditionally employed numerous 
techniques for gathering intelligence information to guide their activities 
(which increasingly may involve sensors, imagery from satellites and 
unmanned air vehicles, or human intelligence sources). They also create 
detailed plans for sending out scouts, place listening and observation posts in 
key locations, and have established procedures for radio and networked 
communication among distributed forces. 

Strater and colleagues (Strater, Endsley, Pleban, & Matthews, 2001a) 


undertook a series of studies to determine differences in SA between 


inexperienced and experienced army platoon leaders. The platoon leader is 
typically the entry-level officer position, thus its new officers are considered 
to be novices in the army hierarchy. Their study involved 14 infantry officers 
including seven lieutenants and seven captains. None of the lieutenants had 
prior experience serving as platoon leaders, although they had just completed 
their initial platoon leader training course, whereas all of the captains were 
experienced in serving as platoon leaders. 

In the study, each participant conducted two missions in a virtual reality 
simulator with a combination of live and digital teammates acting against a 
scripted threat. The actions of the participants were recorded and their SA 
was measured at three points during each scenario using the Situation 
Awareness Global Assessment Technique (SAGAT). At each freeze point, 
the accuracy of their assessment of the situation was measured on 13 
different queries. In addition, an experienced army officer rated the behaviors 
of the participants using the Situation Awareness Behavioral Rating Scale 
(SABARS). 

The SAGAT scores of the two groups are shown in Figure 37.5, with the 
more experienced group (captains) demonstrating higher levels of SA overall 
than the inexperienced lieutenants. These findings show an interesting effect 
of experience on platoon leader SA. Although more experienced officers 
demonstrated superior Level 1 SA in identifying the locations of both their 
own troops and enemy troops, as would be expected, the more interesting 
story involves the subsequent cognitive processes — the transformation of the 
information into higher-level SA. More experienced officers identified the 
strongest enemy and the highest enemy threat (Level 2 SA) with greater 


accuracy than officers with less experience, while less experienced officers 


demonstrated superior performance at identifying the strongest friendly 


elements. 
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Figure 37.5 SAGAT mean scores by experience level. 


Thus, not only did experienced leaders demonstrate higher levels of SA 
on certain factors, as might be expected, but SA proved to be qualitatively 
different depending on the level of experience. That is, with increasing levels 
of experience, platoon leaders shift their focus from concentrating on friendly 
disposition to focusing more on enemy disposition. 

Other research has also found that experienced officers concentrate more 
on the enemy situation than on the friendly situation (Shattuck, Graham, 
Merlo, & Hah, 2000). Thus, it appears that increasing expertise is important 
not only for helping people to take in the correct information, but also in 
prioritization of internal focus for processing that information to form the 
higher levels of SA. 

In terms of their SA related behaviors, the experienced officers were 
rated higher on the factor of gathering information and following procedures 
(uses assets to effectively assess environment, utilizes a standard reporting 
procedure, identifies key elements, sets appropriate levels of alert, assesses 


information received, gathers follow-up information when needed, monitors 


company net, assesses key finds and unusual events) and on focusing on the 
big picture (communicates key information to squad leaders, communicates 
to squad’s overall situation and commander’s intent, solicits information 
from commanders, monitors company net, asks for pertinent intelligence 
information, communicates key information to commander). 

In addition to improvements in how they distributed their attention, the 
findings indicate that with experience, the platoon leaders are significantly 
improving the skill set associated with gathering information from the 
battlefield environment and communicating with others. Communicating 
information both up and down the echelon is important not only for building 
other teammembers’ SA, but also because such activities prompt the delivery 
of relevant information back to the platoon leader. These skills are critical for 
the development of SA in the infantry environment. 

In a follow-up study, Strater, Jones, and Endsley (2001b) surveyed 43 
individuals (both officers and enlisted) who train new platoon leaders. Issues 
that were rated as a frequent problem by more than 25 percent of the 
respondents are shown in [able 37.1. Problems in communication posed a 
major problem for new platoon leaders, ranging from not requesting 
information to not communicating key information. In addition there were 
significant problems reported in gathering information on the combat 


readiness status of the opposing force and one’s own troops. 
Table 37.1 SA problems for new platoon leaders (% reporting). 


Level 1 SA: Failure to correctly gather/detect the critical information 
in the situation 


Not detecting information due to attentional narrowing 27% 


Not utilizing a standard reporting procedure 30% 
Not carrying out standard operating procedure 28% 
Poor intelligence information due to: 
Not requesting pertinent intelligence 31% 
Not employing squads tactically to gather needed information 30% 
Not determining reliability/timeliness of intelligence information 26% 


Poor communication caused by: 


Not requesting information from squad leaders 30% 
Not requesting information from commander 30% 
Not communicating key information to commander 35% 
Not communicating key information to squad leaders 30% 
Not communicating key information to other platoons 44% 
Not monitoring company net 28% 
Not communicating overall situation/commander’s intent to 28% 
squads 


Not determining own combat readiness status 
Experience and training 26% 
Timing/location of direct/indirect fire support 30% 


Not determining combat readiness status of opposing forces 


Number and severity of casualties 

Physical fatigue 

Mental fatigue 

Movement and current position of troops 

Weapons types, characteristics, and quantities available 
Location of direct/indirect fire support 

Ammunition and supplies availability 

Availability of reinforcements 

Heavy weapons location 

Past behavior and tactics 


Impact of current and future weather factors 


Level 2 SA: Failure to comprehend the situation (although basic 
information is detected) 


Not assembling bits of information together to form a coherent 
picture 


Not specifying alternate/supplemental plans/courses of action 


Not developing an understanding of: 


Task priorities 


Impact of soldier load and distance traveled on troop fatigue 


Positioning soldiers to minimize the risk of fratricide 


37% 


30% 


31% 


28% 


33% 


44% 


33% 


37% 


40% 


26% 


26% 


29% 


32% 


33% 


33% 


25% 


Enemy strengths and weaknesses 29% 
Likely areas of strategic significance to enemy 27% 
Enemy expectations of friendly actions 34% 


Level 3 SA: Failure to project the future situation (though current 
situation is understood) 


Lack of contingency planning 39% 


Failure to project the following: 


Usage rate of ammunition and supplies 36% 
Likely enemy COA from available information 33% 
Location of enemy troops around heavy weapons 32% 


Failure to effectively perform the necessary mission tasks 


Poor mission planning 27% 
Poor responses to unexpected/unplanned events 36% 
Poor time management 45% 
Poor task prioritization 28% 


Comprehension problems were noted: failing to assemble bits of 
information together into a coherent picture and not specifying alternate 
courses of action (COAs) were frequent problems for SA, along with not 


understanding task priorities, the impact of load and travel on fatigue, and the 


importance of soldier positioning to minimize fratricide. A continuation of 
Level 1 SA problems was noted with regard to detecting information about 
the enemy. Instructors rated understanding enemy strengths and weaknesses, 
likely areas of strategic significance to the enemy, and enemy expectations of 
friendly actions as major problem areas for SA in new platoon leaders. 

With regard to Level 3 SA, lack of contingency planning was identified 
as a problem area, as was failure to project the usage rate of ammunition and 
supplies. Problems with SA regarding the opposing force were also found at 
the projection level, as trainers noted that new platoon leaders have difficulty 
projecting a likely enemy course of action, as well as their disposition around 
heavy weapons. 

Like the inexperienced pilots, inexperienced platoon leaders appear to 
have many difficulties with forming good SA. Poor skills for gathering 
information are particularly problematic in this domain as very little 
information presents itself otherwise. It was found that the inexperienced 
officers often were not communicating key information because they 
assumed that others knew what was going on, or that the information did not 
need to be passed on. 

These studies also indicate that novice platoon leaders suffer greatly 
from not having good mental models or schema. They are quickly 
overwhelmed by information, are slow to grasp which information is 
important, and do not know where to look for important follow-up 
information. Without these mental models and schema, they also fared poorly 
in integrating information to understand its significance and in projection and 


contingency planning. 


SA and Expertise in Driving 


Driving provides fertile ground for SA research because of its ubiquitous 
nature in modern society. Drivers of various levels of expertise are readily 
available (although experience is often confounded with age in the general 
population). A number of researchers have performed studies examining 
hazard awareness, a form of Level 3 SA, in drivers at various levels of 
expertise. This research determines how far in advance drivers will anticipate 
or project a potential road hazard. These studies have shown a significant 
negative correlation between the lead time for predicting a hazard in a 
simulated driving task or traffic video and drivers’ reported accident rates 
(Currie, 1969; McKenna & Crick, 1991, 1994; Pelz & Krumpat, 1974). This 
finding holds even after age and miles driven are controlled for in the 
analysis (McKenna & Crick, 1991; Quimby, Maycock, Carter, Dixon, & 
Wall, 1986). 

In using this approach to study expertise in drivers, McKenna and Crick 
(1991) found that experienced drivers (more than ten years of experience) 
reacted significantly faster to hazards than novice drivers (less than three 
years of experience), and detected significantly more hazards. When they 
compared drivers with the same number of years of experience, half of whom 
were considered experts (class 1 police drivers in the UK who had completed 
an advanced driving course that included hazard awareness training), they 
again found an advantage for those with more expertise as provided by the 


advanced training. 


Horswill and McKenna (2004) examine possible reasons for the superior 
hazard awareness of experienced drivers. While they acknowledge that some 
of the effect may be due to a change in response bias with experience (i.e. a 
lower response threshold for indicating that something is a hazard), they 
present compelling evidence that this factor cannot explain all of the 
differences associated with expertise. They cite evidence that novice drivers 
tend to focus closer to the front of their vehicle, use their mirrors less 
frequently, and fixate in smaller areas compared with more experienced 
drivers. In addition, more experienced drivers are found to better adjust their 
scanning patterns to the road type (Mourant & Rockwell, 1972) and have 
better search models regarding where to look for hazards (Underwood, 
Chapman, Bowden, & Crundall, 2002). Horswill and McKenna conclude that 
“experienced drivers are conducting a more efficient and effective search for 
hazards rather than simply lowering their criterion for what constitutes a 
hazard” (2004, p. 164). 

In addition, McKenna and his colleagues demonstrate that hazard 
awareness is a cognitively demanding task, and not one prone to high levels 
of automaticity. The superior hazard detection performance of experts suffers 
from exposure to a concurrent memory task (McKenna & Farrand, 1999). 
Although the physical vehicle control portions of the driving task may 
become automatized, the cognitive activities involved in predicting hazards 
appear to continue to require significant cognitive resources. This is 
consistent with reports from pilots that projection takes up a considerable 
portion of their discretionary time. McKenna and colleagues furthermore 
conclude that when the driver’s attention is redirected to other tasks (e.g. 
cellphone usage), they lose much of the advantage that their superior 


projection skills provide. 


Although this body of research focuses primarily on Level 3 SA for the 
driving task, it provides a rich set of data showing that expertise and SA are 
tightly linked in ways that extend well beyond the simple intake of 
information. Whereas other measures of driving (e.g. vehicle control skills) 
have failed to effectively discriminate real-world driving performance (i.e. 
the likelihood of an accident), hazard awareness has been found to have 
significant predictive ability. Hazard awareness is now incorporated as a part 
of the driving test in Australia and the UK (Horswill & McKenna, 2004). 

Similar to the research from the aviation and military domains, this body 
of work finds significant problems with efficient intake of information in less 
experienced drivers. In addition, it appears that SA forms a central and 
conscious task for expert drivers, who may be freed up to develop SA by 
improved automaticity on physical tasks, and enabled by effective mental 
models for organizing and directing information search and interpretation. 

More recent work has found a significant negative effect of automation 
on SA in driving. When the automation is more reliable and more robust, 
people redirect their attention to competing tasks such as daydreaming, 
operating in-vehicle technologies, talking or texting on cellphones, eating, 


grooming, or other extraneous tasks which directly lower their SA (Carsten, 


Lai, Barnard, Jamson, & Merat, 2012; Hergeth, Lorenz, Vilimek, & Krems, 
2016; Kaber & Endsley, 2004; Ma & Kaber, 2005; Sethumadhavan, 2009). 
Merat and colleagues (Merat, Jamson, Lai, & Carsten, 2012) found that 
drivers using autonomous systems redirected their attention to secondary 
tasks, which made them much slower in responding to critical incidents, 
particularly during periods of underload. Other work has found SA to 
increase with automation, as it freed up resources for attending to traffic; 


however, this benefit went away when secondary tasks were introduced 


(Beller, Heesen, & Vollrath, 2013). How new forms of automation and the 


use of in-vehicle technologies will affect novice drivers in particular has not 
been thoroughly examined, but it is expected that they will be even more 
negatively impacted than experienced drivers due to their increased reliance 
on working memory and overall less well-developed long-term memory 


structures for directing attention to the environment. 


Conclusions 


Novices have significant problems in building SA. They suffer from poor 
information management strategies including poorly directed information- 
seeking behaviors and scan patterns that will allow them to detect the most 
important information from amongst the large number of possibilities. Their 
ability to process the information they perceive and to understand its 
significance to their goals is quite limited. They may often fail to appreciate 
the importance or meaning of even the information that they do acquire. In 
that their cognitive resources are often overloaded with carrying out the 
necessary psychomotor and communications tasks, they remain significantly 
behind the ball in developing needed SA. 

By contrast, those with increasing levels of expertise exhibit superior 
strategies for gathering information, both proactively and as a follow-up to 
information already received. They show superior abilities to quickly grasp 
the significance of information and to project what is going to happen, 
allowing them to be in the right place in the sports arena or on the battlefield, 
to have solutions ready to execute in the cockpit, or to avoid road hazards 
while driving. While the act of understanding and projecting can become 
quick and often effortless, with a clear comprehension of the situation 
springing readily to mind, evidence also shows that experts expend 
considerable effort on the task of situation assessment. Actively projecting 
and planning for contingencies forms the hallmark of expertise in SA and 


dynamic decision making. 


It should be noted that much of what has been said here about the role of 
expertise on SA has much in common with the more general literature on the 
effect of expertise on cognitive performance (e.g. see Ericsson & Lehmann, 
1996). SA is not a new cognitive construct that has not been a part of 
cognition in other research. Rather SA has been integral to many of the 
domains in which expertise has been studied historically, even if it has not 
been specifically identified as such. 

Rather than evolving from psychology laboratories, historically the term 
“situation awareness” arose naturally from domain experts in aviation who 
worried about “getting SA” or trying to “keep SA,” dating back as far as 
World War II (Press, 1986). Different terms associated with the same 
construct have been found in other domains as well. Air traffic controllers, 
for instance, talk about “the picture” with an almost identical meaning. 

The case for SA as an area of study is that it provides a focal point 
around which experts integrate the information they gather in order to 
perform their tasks. This integration and future projections, made possible by 
the mechanisms underlying their expertise, are critical to performance in 
these complex and demanding domains. By considering SA as a more holistic 
cognitive construct, therefore, a richer picture of expert performance can be 
generated, and a more focused approach to developing systems that support 
SA becomes possible. 

Expertise undoubtedly also involves other skills, including prowess at 
physically carrying out the needed tasks (for example, throwing a ball, 
driving a car, or flying an airplane) and forming effective decision strategies. 
In addition, the ability to develop and maintain SA is a significant contributor 


to the high levels of performance exhibited by experts in the many domains 


where dynamic situations must be mastered and understood in order to 


perform well. 
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Traditionally, individual differences in the performance of professionals have 
been explained by an account given by Galton (1869/1979; see Ericsson, 
2003). According to this view, there are several factors that influence the 
level of achievement. First and foremost, extensive experience of activities in 
a particular domain is necessary to reach very high levels of performance. 
Extensive experience in that domain does not, however, invariably lead to 


expert levels of achievement. When individuals are first introduced to a 


professional domain after completing their basic training and formal 
education, they often work as apprentices and are supervised by more 
experienced professionals as they accomplish their work-related 
responsibilities. After months of experience they typically attain an 
acceptable level of proficiency, and with longer experience, often years, they 
are able to work as independent professionals. At that time most 
professionals reach a stable, average level of performance, and then they 
maintain this modest level for the rest of their careers. In contrast, some 
continue to improve and eventually reach the highest levels of professional 
mastery. 

This general view is also often invoked to explain age-related declines in 
professional achievement due to the inevitable degradation of general 
capacities and processes with age. More recently, researchers of expertise and 
expert performance have found it necessary to more precisely measure the 
superior performance and then study it under controlled conditions so the 
mechanisms (innate capacities and acquired skills) mediating the superior 
performance can be identified and verified with experimental methods. Once 
these mechanisms have been identified it is possible to examine whether they 
were gradually acquired or innately given, and if they were gradually 
acquired which types of experience and practice have had different effects on 
the continued acquisition and maintenance of an individual’s performance 
(Ericsson, 1996, 2002; Ericsson, Krampe, & Tesch-Rémer, 1993). The expert 
performance framework (Ericsson, 2006, 2015; Ericsson & Ward, 2007) 
proposes that some types of experience, such as merely executing the 
behavior proficiently during routine work without the intention to improve, 


may not lead to further improvement in objective performance, and that 


further improvements depend upon deliberate efforts to change particular 
aspects of performance. 

This chapter updates the review of the effects of experience and 
different types of practice in the first edition of this handbook (Ericsson, 
2006). It will first discuss how expert performance can be described during 
its development. Then it will describe the expert performance approach to the 
study of the development of expert performance, which focuses on the 
mechanisms that mediate the superior performance and their development in 
response to practice. The chapter concludes with a discussion of individual 
differences in the development of expert performance and associations with 


different types and amount of practice. 


Toward Measurement of Domain-Specific 
Reproducibly Superior Performance 


The assessment of expertise has been traditionally based on peers’ and other 
people’s perceptions of general superiority in a domain. Only more recently 
have there been calls for determining what specifically an expert can do that 


other individuals cannot (Ericsson, 2009; Ericsson & Charness, 1994). These 


calls even raised questions about whether peers can accurately assess superior 
outcomes of experts or if they even have the necessary first-hand experience 
of the experts’ relevant performance to do so. For example, many 
professionals, such as doctors, surgeons, and teachers, work independently 
without peers being able to observe how they function and witness the quality 
of their outcomes. When peers describe or nominate someone as an expert, 
they are likely to rely on secondary and non-specific information, such as 
length of experience or presentations and performances at public events. It is 
also important that the studied tasks really capture the essence of the 
expertise that individuals regularly execute in their domain of expertise rather 
than measuring the performance on an available type of task for which the 
individual has not trained. For example, most pianists are only trained to 
perform memorized pieces and therefore it is not a valid measure of their 


music performance to test their ability to play pieces of music without 


preparation (sight-reading) (Lehmann & Ericsson, 1993). It is commonly 
assumed that general abilities, such as intelligence, are predictive of 
individual differences in performance both at the introduction to a domain as 


well as at the advanced stages of expert performance; however the few 


studies correlating performance of groups of experts and their performance 
on general ability tests do not support this assumption (Ericsson, 2014a). It is 
challenging to study how a given type of performance changes across the 


development of an expert performer. 


Tracing the Development of Performance 


Many types of measures of expertise and superior expert performance can 
only be assessed for highly skilled and expert individuals. When a musician 
releases a highly praised recording, a scientist publishes a widely cited article 
in a prestigious journal, or an athlete wins a medal in the Olympic Games we 
see evidence for superior performance, but these measures do not permit 
detailed longitudinal assessment of performance. How would we be able to 
measure the gradual improvement of performance of a potential future 
scientist during childhood, adolescence, and early adulthood that might have 
been necessary for them to attain the adult expert performance? For 
approaches that view expert performance as constrained by some superior 
innate capacity, there has traditionally been much less interest in the 
development of proficiency. For approaches that propose a very important 
role for gradual acquisition of mechanisms an important focus has been on 
how this acquisition can be facilitated and accelerated (Hoffman et al., 2014). 
The approaches also make major efforts to collect data on changes in the 
superior target performance and associated changes in its mediating 


mechanisms as will be shown in the next section. 


Measuring Individual Performance in Domains with Treatment 
Outcomes 
In domains with public, fair competitions, such as music, ballet, sports, and 
games, it is relatively easy to identify individuals with reproducibly superior 


performance (Ericsson, Charness, Hoffman, & Feltovich, 2006). In contrast, 


expertise involving treatment of humans, such as K-12 education, 
psychotherapy, and surgery, is much more difficult to measure and to identify 
objective consequences of reproducibly superior performance by individual 
performers (see Stigler & Miller, Chapter 24, this volume). In these domains 
it is possible to measure the objective performance by the improvements and 
outcomes of students and treated patients. 

In education it is possible to measure students’ performance on 
standardized tests before a given teacher is assigned to teach a particular class 
of students (pre-test). The same students being taught by this teacher can be 
tested on standardized tests (post-test) again a year later. By calculating the 
students’ improvement of their performance, the added value induced by a 
year of instruction by their teacher can be assessed. The average 
improvement of all the students of a given teacher can then be computed and 
compared to the improvements recorded by other teachers with similar 
teaching responsibilities to derive an objective measure of a given teacher’s 


relative performance (Hanushek & Rivkin, 2010). 


It is possible to measure the performance of psychotherapists by 
calculating the difference between patient ratings at the start and end of 
therapy and then comparing outcomes among therapists (Saxon & Barkham, 


2012; Tracey, Wampold, Lichtenberg, & Goodyear 2014). In some surgical 


specialties, such as hand surgery (Waljee & Curtin, 2014), patient satisfaction 
ratings of outcomes are viewed as the criterion measure for assessing the 
success of surgical treatments. Recently, however, there have been several 
instances in medicine where patient satisfaction ratings are not correlated 
with objective indicators of surgical outcomes or process measures of quality 


of surgical procedures (see Ericsson, 2015). Consequently, whenever possible 


researchers should identify objective measures of patient outcomes. 
Another challenge to the measurement of expert performance of 
individuals is that they often perform in the context of groups and teams and 


this challenge will be discussed next. 


Expert Performance of Teams and Groups 


Another issue concerns assessing the expertise of members of teams, such as 
in American soccer, football, and baseball or among orchestras and choirs 
(see Sonesh, Lacerenza, Marlow, & Salas, Chapter 27, this volume). The 
standard approach has been to assess the performance level of the team by the 
performance level at which the team competes or performs publicly at local, 
district, state, national, or international levels. The assumption is that coaches 
of successful teams will recruit individuals who can perform at the current or 
desired level. In most teams there are different roles and playing positions, 
such as the quarterback and the kicker in American football, where the 
particular individual attributes and training background might be very 
different. 

Even when many individuals perform collaboratively in teams, it is 
possible to observe, measure, and analyze each of the individuals’ 
performances. The individual performance of orchestra members, for 
example, can be elicited by having the member play by themselves as is done 
during auditions for jobs in orchestras. If one is interested in testing the 
musician’s ability to perform as a member of an orchestra, the performance 
can be generated while listening to a music recording. The evaluation of 
music performance can be accomplished by several independent judges 
listening to the taped performance without any knowledge of the musician(s) 
(Ericsson et al., 1993). Individual analyses of performance become more 
difficult when sports teams play matches against other teams or surgical 
teams complete operations and post-surgical care with patients. In both cases, 


it is possible to record the events with video cameras and then analyze the 


video record and other objective evidence to assess the accuracy and success 
of each team member’s actions. These analyses will consider each 
individual’s role and playing position within the team. For example, in soccer 
a midfielder might be evaluated by the percentage of successful passes to 
offensive teammates and a center in basketball might be evaluated by the 
percentage of successful passes to other players and of successful scoring for 
all ball possessions. Success of actions will depend on the level of 
performance of the opposing players and thus can only be evaluated at a 
particular level of competition, such as during matches in a national league. It 
is also possible to search for tasks conducted as a team, where the primary 
objective outcomes can be linked to one member of the team. For example, 
the recurrence of a cancer after cancer surgery has been linked to the skill 


level of the primary surgeon (Vickers et al., 2007). 


The Expert Performance Framework 


The focus of the expert performance approach is to identify reproducibly 
superior individual performance in representative tasks under controlled 
circumstances. In order to demonstrate that the performance of a given 
individual is superior to that of other performers it is nearly always necessary 
to find comparable tasks that other people perform regularly during their jobs 
or public performances and competitions. Only when many individuals 
complete the same or similar representative tasks can their objective 
performance be directly compared (Ericsson & Smith, 1991; Ericsson & 
Ward, 2007; Ericsson & Williams, 2007). According to the expert 


performance approach it is essential to be able to reproduce the superior 


performance or aspects of that performance under controlled conditions so 
the mechanisms mediating that performance can be _ identified by 


experimental methods. 


Identification of Mechanisms Mediating Captured Superior 
Performance 

The measurement of reliable individual differences among experts’ and 
professionals’ performances in particular domains requires aggregating, say, 
the effects of hundreds of hours of competition in chess tournaments, of 
numerous hours of surgery on hundreds of patients, and of many hours of 
teaching hundreds of students in the classroom. Even if all these hundreds of 
hours had been recorded by video it would be nearly impossible to identify 
which specific behaviors or actions might account for the observed individual 
differences in average performance and treatment outcomes. 

The activities of experts and professionals are not likely influencing the 
crucial outcomes equally. For example, during a surgery procedure lasting 
several hours much of the time is spent on preparing for the major repair or 
removal of damaged tissue. This lengthy preparation would be more routine 
and thus less likely to influence patient outcomes than the surgical repair and 
the handling of any emergency situations that were encountered 
unexpectedly. Similarly, in a chess game among skilled or expert players, not 
every move is equally important for the outcome of the game. For example, 
in chess the first few moves in a chess opening are typically recalled from 
memory. There are also many middle game positions for which no clearly 
superior move can be determined even by the best players in the world. There 
are, however, some decisions in a game where the consequences of making 
one move rather than another will clearly influence the probability of winning 
the game. By recreating these associated chess positions and instructing 


participants differing in skill in a domain to generate their moves for these 


positions in real time, one should be able to capture the performance 
differences and the associated processes mediating superior performance by 
observing the individuals for a much shorter time than observing entire 


games. I will illustrate this method in a few different domains. 


Decision Making in Chess and Other Domains 


Chess players’ skill levels can be determined by the outcomes of 20—40 
matches in chess tournaments against opponents with different skill ratings 
(Elo, 1978). The corresponding chess playing required to complete that 
number of chess matches is over 100 hours of chess playing or over 1,000 
chess moves. All chess moves are not equally important for outcomes of 
chess games, and a pioneering study by de Groot (1978) identified several 
critical chess positions taken from real games and asked world-class and club 
players to think aloud while they tried to pick the best move for each 
position. Subsequent research has assembled chess positions with critical 
moves and instructed chess players to try to select the best move for these 
positions. When chess players attempt to select the best move for a couple of 
dozen positions, they can complete their selections within 15-20 minutes, 
and their average score is highly correlated (r around 0.8) with their official 
chess rating for chess players ranging from novices to grand masters (van der 


Maas & Wagenmakers, 2005). With the emergence of chess playing 


computer programs, which are vastly superior to human players in their move 
selection, it is now possible to get objective scoring of different moves for 
critical chess positions (Guid & Bratko, 2011). 

Chapter 12 in this volume, on capturing expert thought with protocol 


analysis, describes a number of applications capturing superior performance 


in games, such as poker and Scrabble, in medicine, such as diagnosing 
medical images, and in sports, such as selecting the appropriate action in 


situations displayed by videos from games. 


Extended Activities, such as Running and Climbing 


It is nearly impossible to follow a runner in a marathon or a mountain climber 
reaching the summit of a mountain. One of the first successful efforts to 
capture reproducibly superior performance in the laboratory involved 
building treadmills, where runners could run within a laboratory setting or 
indoor training facility. Early laboratory studies revealed that the best 
predictor of actual running performance in races for expert runners was the 
running economy on a treadmill, namely, the amount of oxygen necessary to 
maintain normal race pace, rather than the maximal consumption of oxygen 
(vO*,nax) (Saunders, Pyne, Telford, & Hawley, 2004). Similarly, with the 


recent emergence of indoor climbing gyms it is now possible to study the 


decision making about climbing paths as well as the mechanical ability to 
climb. One of the most striking measures of climbers and their ability to 
climb difficult ascents is the strength of their fingers. The ability to hang from 
only the tips of one’s fingers is highly correlated (R**2 > 0.65) with 
successful climbing of mountains by ascents of increasing difficulty (Bala§, 


Pecha, Martin, & Cochrane, 2012). The ability to hang from their fingertips 


appears to be a necessary ability, but obviously not a sufficient ability for 
climbing mountains and therefore this test is not a representative task for this 
type of expertise. In controlled and safe environments it is possible to 


measure differences in performance on particular types of tasks that 


differentiate individuals at different levels of skills and might suggest 
different practice histories. 

There are many demonstrations of superior accuracy and reproducibility 
of motor performance under laboratory conditions. For example, the more 
skilled bowlers in cricket are less variable in executing a given bowling 
action and can better control outcomes over delivery types with different 
landing positions (Phillips, Portus, Davids, & Renshaw, 2012). Expert 
pianists are more able to reproduce the exact timing and key pressures when 
playing the same piece several times than amateur pianists (Krampe & 
Ericsson, 1996). 


Summary 


The focus on reproducibly superior performance seeks to capture the 
superiority in terms of performance levels for individuals on standardized or 
representative tasks, and then once experts are identified, research can 
investigate the mechanisms that mediate this performance, in as much detail 
as possible. For each identified mechanism the expert performance approach 
raises the question of how the associated differences emerged through 
development and whether they were influenced by training or determined by 
innate genetic differences. Our knowledge of the development of expert 


performance will be reviewed in the next section. 


Development of Reproducibly Superior 
Performance 


Virtually all frameworks for studying expertise and expert performance 
recognize that development during childhood and adolescence is necessary as 
a basis for attaining the highest level of achievement. The highest levels of 
performance can then be attained after reaching physical maturity (around 
age 18) in most domains of expertise. Reviews show that extended 
engagement in domain-related activities is nearly always necessary to attain 


expert performance in that domain (Ericsson, 1996; Ericsson & Lehmann, 


1996). In some domains individuals differing in age have their objective 
performance measured in an age-independent manner, such as chess ratings 
or running times for races of fixed length. In these domains the development 
of expert performance as a function of age and years of experience can be 
described, as follows. First, longitudinal assessments of the same individuals 
reveal that individuals’ performances improve gradually, as illustrated in 
Figure 38.1. There is no convincing evidence, as far as I know, that a child or 
adult is able to exhibit a high level of performance without any relevant prior 
experience and practice. Similarly, there is a lack of evidence for abrupt 
improvements of reproducible performance when it is assessed on a regular 
basis. Second, peak performance of experts is nearly always attained in 
adulthood — many years, and even decades, after initial exposure to the 
domain, as illustrated in Figure 38.1. The age at which performers typically 
reach their highest level of performance in many vigorous sports is the mid- 


to-late 20s (Schulz & Curnow, 1988). For the arts and science, it is around a 


decade later, in the 30s and 40s (Simonton, 1997). The continued and often 
extended development of expert achievement past that age of physical 
maturity suggests that additional years of experience are necessary to attain 


the highest level of performance. 
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Figure 38.1 An illustration of the gradual increases in expert performance 
as a function of age, in domains such as chess. The international level, 
which is attained after more than ten years of involvement in the domain, 


is indicated by the horizontal dashed line. 


From K. A. Ericsson and A. C. Lehmann, “Expertise,” Encyclopedia of 
creativity. © Academic Press, 1999. 


Finally, perhaps the most compelling evidence for the important role of 
experience for developing expertise comes from longitudinal studies that 
have shown that, even for the most talented individuals, a minimum of ten 
years of experience in a well-established domain (ten-year rule) seems to be 


necessary but not sufficient to become an expert (Bryan & Harter, 1899) and 


to win consistently at an international level in chess (Simon & Chase, 1973). 


These studies are easier to conduct as they only identify two ages, namely the 


age of first introduction to the domain and the age at which the international 


performance was attained without detailed information of the intermediate 
performance and training during the intervening years. Subsequent reviews 
have compared starting age with a performer’s age in reaching an 
international level of achievement and have shown that these estimates of 
prerequisite experience related to the domains extend to international level 
success in composing music (Hayes, 1981), as well as to sports, science, and 
the arts (Ericsson et al., 1993). More recent analyses of the evidence for the 
ten-year precedent show that the number of years of intense training required 
to become an internationally acclaimed performer differs across domains. For 
example, there is evidence that some recent grand masters in chess were able 
to reach that level several years faster than the hypothesized ten-year limit, 
possibly due to better training resources (Gobet, Campitelli, & Waters, 2002). 
At the other end, elite musicians need closer to 20 to 30 years of training and 
often peak when they are around 30-40 years old. Further, outstanding 
scientists and authors normally published their first work at around the age of 
25 (after some 20 years of school and domain-related experience) (Bloom, 
1985a), and their best work follows around ten years later (Raskin, 1936). 
This general evidence suggests that expert achievement requires extended 
experience beyond attained physical maturation in the form of continued 
engagement in different types of practice. In contrast, accounts based on 
innate individual differences in talent would argue that the potential for talent 
remains stable and its expression develops with maturation in a manner that 
depends primarily on age. Consistent with innate talent, this pattern is well 
illustrated by a small number of attributes, such as the developmental pattern 
of height and body size. 

Within the expert performance approach there has been an effort to 


identify the same measures of performance throughout the development of an 


individual’s performance, such as running times for 400 m, and accuracy of 
shooting rifles at 100 m. Collecting longitudinal measurements permits a 
direct comparison of performance improvement at different ages during the 
career development. It should be possible, at least in principle, to describe the 
development of each individual’s performance as an ordered sequence of 
stable states of performance. Each state can be described in terms of the level 
of attained performance. For example, in domains such as music, gymnastics, 
and ballet, various types of performance are graded in terms of their 
complexity and there are curricula that recommend a particular order for 
mastery of progressively more complex performances. More specifically, 
piano students start with basic techniques and after a couple of years start 
mastering more difficult techniques involving their non-dominant hand; 
however, only after some ten years of piano instruction do they start 
mastering polyrhythms where the two hands play with different rhythms, 
such as four beats in one hand against five beats in the other during the same 
time interval (Lehmann & Ericsson, 1998). 

By measuring the dates and level of mastery for individuals it would be 
possible to record trajectories of developed performance for individuals. 
There are clear individual differences in attained performance at various ages. 
Prodigies appear to follow the same trajectories as other individuals, but they 
start at younger ages and progress faster to more advanced levels (Ericsson, 
20146; Ericsson, Roring, & Nandagopal, 2007; Feldman, 1994). Research on 


soccer players has shown that performance on certain tests at particular ages 
is correlated with attaining the highest levels as adult players (Huijgen, 
Elferink-Gemser, Post, & Visscher, 2010). Most interestingly, the types of 
performance tests that yield predictive scores differ at different ages during 


late adolescence and early adulthood. The goal of the expert performance 


approach is to describe and measure the gradual changes in performance and 


attempt to specify the changes in the mediating mechanisms. 


Accounting for Individual Differences in the 
Development of Expert Performance 


The best example of an attribute that is critical to particular domains of 
expertise and that seems to be virtually immune to influences of training and 
experience is height. In domains where greater height is important 
(basketball) or shorter height is beneficial (artistic gymnastics for women) it 
is difficult to predict participants’ height as adults, when they start training as 
children. A child’s height is a relatively poor predictor of their height as 
adults. To deal with this problem, coaches typically collect information about 
the parents’ height and maturation status and are able to predict adult height 


accurately for children from age 8 onwards (Ostojic, 2013). Height is related 


to the length of bones, which seems to be unresponsive to training, but many 
other aspects of physiology and anatomy, such as width of bones, size of 
heart and arteries, type of metabolic processes of muscles, and myelinization 
of nerve fibers have been shown to adapt to particular types of training and 


experience (Ericsson, 2007a). It is possible that future research will discover 


other physiological and anatomical attributes that cannot be changed by 
training yet are highly related to success in a domain of expertise; however, 


to date, the discoveries of such variables have been rare. 


Limited Effects of Accumulated Experience on Attained Level of 
Performance 
When individuals are first introduced to an everyday or recreational activity 
such as driving a car, typing on a computer, or playing golf, their primary 
goal is to reach a level of proficiency that will allow them to perform these 
tasks at an acceptable level. At the first encounter with the activity, beginners 
try to understand the requirements of the activity and focus on avoiding gross 


mistakes (Anderson, 1982; Fitts & Posner, 1967; black area in Figure 38.2). 


With more experience, noticeable mistakes become increasingly rare, 
performance appears smoother, and learners no longer need to focus as 
intensely on their performance to maintain an acceptable level (gray area in 
Figure 38.2). After a limited period of training and experience — frequently 
less than 50 hours for most everyday activities such as typing (Dvorak, 
Merrick, Dealey, & Ford, 1936) and driving a car (Groeger & Brady, 2004) — 


an acceptable level of performance is typically attained and individuals are 


able to execute these skills smoothly and with minimal effort (white area in 
Figure 38.2). Similar findings have been observed for limited training with 
laboratory tasks (Ericsson et al., 1993). During this automated phase, 


performance reaches a stable plateau, where no further improvements are 


typically observed (see Ericsson, 2009, for a review). There are numerous 


recreational golf and tennis players whose performance has not improved in 


spite of decades of weekly playing. 
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Figure 38.2 An illustration of the qualitative difference between the course 
of improvement of expert performance and of everyday activities. The goal 
for everyday activities is to reach as rapidly as possible a satisfactory level 
that is stable and “autonomous.” After individuals pass through the 
“cognitive” and “associative” phases they can generate their performance 
virtually automatically with a minimal amount of effort (see the gray/white 
plateau at the bottom of the graph). In contrast, expert performers 
counteract automaticity by developing increasingly complex mental 
representations to attain higher levels of control of their performance and 
will therefore remain within the “cognitive” and “associative” phases. 
Some experts will at some point in their career give up their commitment 
to seeking excellence and thus terminate regular engagement in deliberate 
practice to further improve performance which results in premature 


automation of their performance. 


Adapted from K. A. Ericsson, “The scientific study of expert levels of 
performance: General implications for optimal learning and creativity,” 
High Ability Studies, 9, p. 90. © European Council for High Ability, 
1998. 


Comparable phases of acquisition have been documented in professional 


domains such as telegraphy (Bryan & Harter, 1899) and typing (Book, 1925) 


with evidence for plateaus, where performance is not increased with 
additional experience. Similarly the performance of highly experienced 
individuals is often found to be comparable to that of less experienced 
individuals in domains of expertise such as medicine (Ericsson, 2007a), 
auditing (Bédard & Chi, 1993), language translation (Jaaskeldinen, 2010), 
nursing (Ericsson, Whyte, & Ward, 2007), and several other domains 
(Ericsson, 2006). Extended professional experience beyond medical or 
nursing school has even been found to be associated with decrements in 
performance, most likely due to forgetting (Choudhry, Fletcher, & Soumerai, 
2005; Ericsson, 2007a). Recent meta-analyses of clinical reasoning showed a 
similar effect — with a noticeable disadvantage for people without any 
experience, but little if any advantage for additional experience beyond the 
initial exposures (Spengler & Pilipis, 2015). Very limited improvements with 
increased professional experience are seen for psychotherapists’ patient 
outcomes (Montgomery, Kunik, Wilson, Stanley, & Weiss, 2010) and with 
teachers’ ability to improve students beyond the first couple of years as 
teachers (Hanushek & Rivkin, 2010). Similarly, experience of playing a 
larger number of chess games on the Internet has only a low yet statistically 
significant correlation (r (469) = 0.17) with chess skill (Ericsson & Moxley, 
2012). In most cases when chess players lose games, they do not know 
exactly why or how they need to change the way they play. There are, 
however, at least some domains where individuals get immediate feedback on 
mistakes and problems, such as during and immediately after surgery, and in 
these domains accumulated numbers of surgeries of a particular kind are 
associated with improved performance (Ericsson, 2015). 

The adaptation to the demands of professional and recreational activities 


is nicely illustrated by physiological characteristics of adults, such as physical 


fitness. Individuals have adapted to physiological demands in their habitual 
activities. It has been known for a long time that improvements of aerobic 
fitness require individuals to change their activity level and engage in intense 
aerobic exercise that pushes them well beyond the level of comfortable 
physical activity (Ericsson et al., 1993; Robergs & Roberts, 1997). 
Specifically, in order to increase their aerobic fitness, young adults have to 
exercise at least a couple of times each week, for at least 30 minutes per 
session, with a sustained heart rate that is 70 percent of their maximal level or 
around 140 beats per minute. When the human body is repeatedly put under 
exceptional extended strain, a range of dormant genes in the DNA are 
expressed and extraordinary physiological processes are activated. Over time, 
the cells of the body, including the brain, will adapt in response to the 
induced metabolic demands of the activity by, for example, increased growth 
of cells in capillaries supplying blood to muscles and changes in cell 
metabolism in the muscle fibers themselves. These adaptations eventually 
allow the individual to continue executing the given level of activity without 
greatly straining the physiological systems. To gain further beneficial 
increases in adaptation, individuals need to increase or change their weekly 
training activities to induce new and perhaps different types of strain on the 
key physiological systems. 

More generally, the training of athletes involves pushing their associated 
physiological systems beyond their comfort zones to stimulate physiological 
growth and adaptation (Ericsson, 2007b, 2014a). In particular, it spurs 
muscular development and blood supply. In addition, there is clear evidence 
that intense training at relatively young ages is necessary for modifying joints 
to make them suitable for ballet and baseball pitching (Ericsson, 2007b). The 


neural connections and circuitry of the brain are also powerfully influenced 


and shaped by training. In recent years, neuroplasticity has been shown to be 
integral to adult learning and development (L6vdén, Wenger, Martensson, 
Lindenberger, & Backman 2013). A recent review (Wei, Zhang, Jiang, & 
Luo, 2011) found systematic differences in the structure of the brains of adult 
experts in basketball, golf, judo, and diving compared to the general 
population, whose differences were most likely the result of extended 
practice (see Bilali¢ and Campitelli, Chapter 14, this volume). In fact, injured 
athletes’ fitness is reduced after weeks of enforced bedrest (Williams, 
Kuipers, Mukai, & Thirsk, 2009). During early space flights well-trained 
astronauts experienced dramatic reductions in their physiological fitness 
during low-gravity space travel until they started to engage in physical 
exercise programs during space flights (Williams et al., 2009). 

In sum, merely performing the same activities over and over without 
immediate feedback on the outcomes does not in and of itself lead to 
improved accuracy of performance — it primarily leads to reductions in the 
level of effort. Under those conditions performance rapidly reaches a modest 
plateau, and does not increase with further experience, as shown in Figure 
38.2. 


Types of Training Mediating Improved Performance for Years and 
Decades 

The improvements for amateurs and experienced professionals are often 
modest even after decades of experience in professional activities or 
engagement in recreational games. In direct contrast, individuals who will in 
the future achieve expertise show marked improvements in performance for 
years and even decades, as shown in Figure 38.1. In a pioneering research 
project Bloom (1985b) and his colleagues studied individuals who had 
reached an international level of achievement in six different domains and 
found that they had more beneficial developmental conditions than their 
peers, in particular involving instruction and supervised training by teachers 
during their childhood and adolescence. 

Sufficient training and engagement has been shown to be necessary to 


reach an international level (Bloom, 1985b). Ericsson et al. (1993) set out to 


explore whether differences in the quality and amount of training could 
account for some of the individual differences among expert musicians. They 
searched for evidence of “conditions for optimal learning and improvement 
of performance” (p. 367), where “[T]the most cited condition concerns the 
subjects’ motivation to attend to the task and exert effort to improve their 
performance ... The subjects should receive immediate informative feedback 
and knowledge of results of their performance. The subjects should 
repeatedly perform the same or similar tasks” (p. 367). Drawing on Chase 


and Ericsson’s (1982; Ericsson, 2013) experiments showing over 1,000 


percent increase in memory performance with practice on well-defined 


memorization tasks that also afford immediate feedback on performance, 


Ericsson et al. (1993) argued that this type of training can be found in 
domains of expertise with a very long history. One such domain is music, 
where aspiring musicians are given individualized instruction and “the 
teacher designs practice activities that the individual can engage in between 
meetings with the teacher. We call these practice activities deliberate 
practice” (p. 368). 

Ericsson et al. (1993) collected data on objective performances on 
representative tasks that captured the essence of expertise in music, such as 
success at music competitions, and compared students with the promise of 
becoming international soloists to two groups of less accomplished violinists 
at the same international music academy in Berlin. The goal of this study was 
to describe different training and practice activities and then identify those 
that had differentially contributed to the violinists’ different levels of attained 
performance. All three groups of violinists identified the time practicing 
alone, on tasks with goals determined at weekly meetings with their teachers, 
as the most relevant activity for improving their performance (among 
activities where the violinists could control their duration). Most importantly, 
this activity was rated as requiring high levels of concentration and effort, 
and not rated as enjoyable as leisure activities with friends. This research 
showed that even at this elite level of performance, the history of the amount 
of solitary practice accounted for significant differences between the groups — 
with most highly accomplished violinists having accumulated more practice 
than the other two groups. 

There is a close correspondence between the conception of the 
development of expertise as a sequence of states of performance (see Figure 
38.3) and deliberate practice as defined as “the individualized training 


activities specially designed by a coach or teacher to improve specific aspects 


of an individual’s performance through repetition and successive refinement” 
(Ericsson & Lehmann, 1996, pp. 278-279). In domains such as music 
training for a particular instrument, there are curricula that recommend a 
progression of changes and mastery of increasingly difficult techniques 
during each year of music instruction (Lehmann & Ericsson, 1998). During 
deliberate practice, a teacher or coach will assess the current level of 
performance (see the state illustrated in Figure 38.3) and identify the next 
goal for performance appropriate for each individual and help the performers 
to engage in a training activity that will allow them to reach the targeted goal. 
To carry out deliberate practice successfully between meetings with the 
teacher, the student needs to schedule their practice and monitor their 
performance during practice to make adjustments to meet the practice goals. 
At the start of music practice for a young child, parents often help their 
children plan practice and help them to identify errors and sustain attention to 
their corrections during subsequent repetitions. With more experience, the 
students are able to acquire three systems of representations, as originally 
proposed by Ericsson (1996). These systems can be easily identified in expert 


musicians (see the resulting state in Figure 38.3). 


Deliberate 
Sa Practice 


Performance 


Deliberate Practice 


Figure 38.3 A schematic illustration of the acquisition of expert 
performance as a series of states with mechanisms of increasing 
complexity for monitoring and guiding future improvements of specific 
aspects of performance. Each state can be described in terms of three 
systems of representations and their interconnections, where the increased 
size of the oval illustrates the corresponding system’s increased 


complexity, refinement, and interconnectedness. 


Adapted from K. A. Ericsson, “The scientific study of expert levels of 
performance can guide training for producing superior achievement in 
creative domains,” Proceedings of the International Conference on the 
Cultivation and Education of Creativity and Innovation (p. 14). Beijing, 
China: Chinese Academy of Sciences. © International Research 


Association for Talent Development and Excellence, 2009. 


The first type of system or mental representation associated with each 
state allows expert musicians to image the sounds of a piece that they want to 
produce while they play it. For example, in preparing for a public concert, 
expert musicians will image what a music piece will eventually sound like 
when it is performed in front of an audience (see System 1 in Figure 38.4). 
The same system allows musicians to image the immediately preceding part 
of pieces during the playing of a particular piece. The second representation 
(System 2 in Figure 38.4) attempts to translate the image in the first 
representation into actions that result in music that can be heard by anyone 
listening to the musicians’ practice or performance. The third and final 
representation (System 3 in Figure 38.4) permits the expert musician to listen 
to what their current performance sounds like as they are playing. A 


discrepancy between the aspired music expression (System 1) and the actual 


music expression (System 3) allows the expert musicians to identify 
differences, which they can reduce by focused practice and then eventually 
produce an approximate realization of the aspired music expression. These 
acquired representations can then be integrated with expanded working 
memory (see Ericsson, Chapter 36, this volume). 
Music “Jmagined desired music experience” 
System | 


Desired 
performance goal 


Music “Playing the piece of music” “Listening to the played music 
as experienced by an audience” 


Figure 38.4 Three types of systems of internal representations that mediate 


expert music performance and its continued improvement during practice. 


Adapted from Figure 6 in K. A. Ericsson, “The scientific study of expert 
levels of performance: General implications for optimal learning and 
creativity,” High Ability Studies, 9 (p. 92). © High Ability Studies, 1998. 


A reasonable translation of the definition of deliberate practice (Ericsson 


et al., 1993) in music to other domains of expertise would be to identify one- 


on-one instruction of a performer by a coach or teacher, who assigns practice 
activities with explicit goals and with immediate feedback and opportunities 
for repetition. Teachers can monitor the students’ attained practice goals and 
how they transfer to improvements in their public performances. This method 
is thus a direct match to the progression toward expert performance according 


to a curriculum associated with the sequence of states as illustrated in Figure 


38.3. This type of individualized one-on-one instruction with assigned 
training tasks is, however, not the predominant practice activity in most 
domains of expertise. Although teachers and coaches may give comments 
and instructions occasionally, it is rare for the coach or teacher to set up a 
training activity where a trainee would be able to practice with immediate 
feedback until they have mastered a given weakness. However, in some 
domains of expertise teachers assign individualized practice tasks with 
immediate feedback during most of the practice time. Rare estimates for 


hours of practice meeting the criteria for deliberate practice are available for 


rhythmic gymnastics (Law, Coté, & Ericsson, 2007), as one of many types of 
different training by international, national, and regional athletes (Baker, 
Coté, & Abernethy, 2003; Young & Salmela, 2010), and for pianists 
(Ericsson et al., 1993) and professional ballet dancers (Hutchinson, Sachs- 


Ericsson, & Ericsson, 2013). 


Other Types of Training Activities 


There are several aspects of deliberate practice that are not present in other 
types of training activities. The first criterion is that the training is guided by 
a teacher or a coach and is designed to help a particular performer improve 
his/her performance by identifying an aspect of the performance that the 
performer is capable of attaining with limited training activity (typically 
proposed by the teacher). The second criterion is that training activity must 
be designed to allow the student to repeatedly execute it with immediate 
feedback to gain gradual improvements. 

In most other domains, training rarely involves individualized training 
designed by a coach or teacher. A common example is the lecture, where a 
teacher presents the same material to dozens or even hundreds of listeners. 
When lectures are given in professional education in medicine, research 
shows very limited influence on daily behavior and performance in the clinic 
(Davis, 2009). Similar results are found in general education (Hattie, 2009) 
and have led to initiatives to individualize training in college (Deslauriers, 
Schelew, & Wieman, 2011). Even in contexts where coaches keep 
individuals actively engaged in behavior in the domain, such as sports teams 
and orchestras, the training is often not individualized and directed instead 
toward keeping all members of the team engaged in a single activity. This 


type of practice we will refer to as group-based practice. 


Group-Based Practice 


Most coach-led practice involves a group of trainees and thus each trainee 


will engage in training activities that are rarely selected for a particular 


individual’s needs. This type of team practice often does not yield 
individualized feedback with opportunities for corrected repetitions on a 
regular basis. Many types of team practice activities involve scrimmages 
between groups of team members or training of groups and choirs. These 
group activities will involve several trainees at a time with limited 
opportunities for detailed and individualized feedback and scarcely any 
opportunities for individualized repetitions for gaining gradual improvement. 
For example, if trainees miss a backhand volley during a doubles game in 
tennis, they are not likely to perform better when the same situation emerges 
unexpectedly and thus more playing experience will not lead to direct 
improvements. In contrast, a couple of sessions with a tennis coach would 
allow the player to practice the volley and gradually gain more control and 
then integrate it into their regular game. In that type of coach-led practice, the 
increase of the tennis player’s backhand volley performance and its transfer 
to real games can be observed and measured. In contrast, the effect on 
performance from team practice is not measured and might not even be 
observable, which raises questions about its effectiveness for improving the 
objective performance of team or group members. Even when individuals 
engage in practice by themselves, this practice rarely meets all the criteria of 
deliberate practice, especially the requirement that it is supervised and 


individualized by a coach or teacher. 


Individualized Practice without a Coach 


In some domains, one-on-one instruction is rare but several domains offer 
practice activities that meet several of the criteria for deliberate practice. 


When individuals train individually in darts, bowling, and chess they can 


design solitary activities that allow for focused practice on some aspect of 
performance by repeatedly executing actions with immediate feedback, such 
as hitting a particular area on the dartboard or selecting a chess move for a 


given chess position. 


Chess and Decision Making 


Chess players typically practice this task by studying chess openings and 
analyzing published games between the very best chess players in the world. 
They can simulate actually playing against world-class players by trying to 
select the best move for each chess position in the game (Ericsson et al., 
1993). Once they have selected their move they can look up the 
corresponding move originally selected by the masters in the book. If the 
master’s move in the studied chess game differed from their own selection, 
this would imply that their planning and evaluation must have overlooked 
some aspect of the position. It is important to note that this learning process 
requires that the player is able to analyze their thinking during the move 
selection to diagnose the source of any suboptimal moves and thus identify 
how future selection of related moves can be improved. This type of 
generation of decisions in response to naturally occurring situations and with 
immediate feedback on the outcome allows performers to diagnose the need 
for changes in their planning and mental representations to lead to improved 


selection of future actions. 


Typing and Increases in Speed 


When professionals work, they try to minimize risks of mistakes and thus try 


to maintain considerable safety margins. For example, in the last century 


there was considerable research on secretaries who did extensive retyping of 
manuscripts with corrections. Research showed that if typists wanted to 
increase their typing speed, they needed to set aside 15-30 minutes outside 
work where they pushed their typing speed to be around 10-20 percent faster 
than their normal speed (Dvorak et al., 1936). With increased typing speed 
they could identify letter combinations that slowed them down. By focused 
training on those challenging combinations these problems can be eliminated 
and new and different problem combinations can be identified and mastered 
with training and so on. Similar methods of pushing performance beyond 
normal are exemplified by interval training for long-distance running 
(Laursen & Jenkins, 2002). 


Control and Reproducibility of Performance 


Performance involving perceptual-motor processes requires a high level of 
control of the outcomes. Throwing darts at a dartboard, bowling on a bowling 
lane, and putting a golf ball provide immediate accurate feedback on the 
outcome of one’s actions and thus provide suitable practice tasks, where 
consistent and controlled actions in different situations can be attained by 
refinement of representations by systematic adjustments in solitary practice. 
When a refined representation of the current situation during dynamic 
activities has been acquired it is possible for expert performers to react 


appropriately and accurately to changing conditions. 


Other Types of Practice Activities 


There is a very long list of activities that aspiring expert performers could 


engage in to attempt to improve their objective performance. For example, 


Young (1998) distinguished 23 activities for middle-distance runners, such as 
keeping a training journal, watching other people run, training focused on 
technique, reading books about running, and weight training for power. Baker 
et al. (2003) identified 11 categories for national and international level 
athletes, such as watching games on television, aerobic training, and video 
training. There is no generally accepted practical method for assessing 
whether engaging in any one of these activities is more or less beneficial for 
attaining higher levels of expert performance in traditional domains. As I will 
show in the next section it is virtually impossible to study the entire period of 


the acquisition of expert performance with experimental methods. 


The Differential Effectiveness of Practice 
Activities to Improve Performance 


Ideally, one would design experiments with randomly assigned control 
groups, where different groups are instructed to engage in different types and 
combinations of practice for 3-5 hours per day and monitor differential 
increases in their objective performance for the next 10—20 years. Even if the 
resources could be gathered to provide full-time support to the selected 
trainees and their teachers, there is no known acceptable method for keeping 
all of the individuals sufficiently motivated to engage in the practice activities 
with full concentration for a decade and more. It is already known that 
engaging in practice with full concentration to gain the maximal benefits is 
not experienced as being as enjoyable as alternative activities involving social 
leisure and play. There are two major methods for assessing the effectiveness 
of different types of practice. It is possible to compare different practice 
methods in short-term experiments lasting for less than 50 hours. The most 
common method involves identification of different types of practice 
activities, where the estimated duration of engagement in each type of 


activity is correlated with attained superior objective performance. 


Relations between Amount of Engagement and Attained 
Improvements of Performance 
In their original study, Ericsson et al. (1993) identified solitary practice as 
one of the 12 music-related activities that they hypothesized as consisting of 
proportionately longer periods of deliberate practice, and they found that the 
amount of current and accumulated solitary practice alone was correlated 
with attained music performance. In contrast, when the sum of engagement in 
any types of music-related activity was compared between groups differing in 
performance there were no statistically significant differences in number of 
hours. When the estimated engagement in one activity is more closely related 
to attained performance or current improvements in performance, it is an 
indicator that this activity is more related to improvement than other 
activities. Other studies have found that the correlation between the amount 
of coach-led individual training and objective performance is substantial, 


approaching 0.9 (Ericsson, 2016). Other studies of the effects of practice 


alone have found the correlation between accumulated serious study in chess 


and chess rating is much higher than the correlations with other chess-related 


activities (Charness, Tuffiash, Krampe, Reingold, & Vasyukova, 2005). 
Similarly, the effect of accumulated practice alone and _ objective 
representative performance is much higher than accumulated estimates of 
engaging in playing with others in domains such as bowling (Harris, 2008) 
and darts (Duffy, Baluch, & Ericsson, 2004). 


All Types of Practice are not Equally Effective in Improving 
Performance 


The concept of deliberate practice implies that individualized practice in 
training tasks recommended by a coach, with immediate feedback, is more 
effective in improving individual performance than watching a soccer match 
on TV or engaging in practice scrimmages of a team. Consistent with this 
hypothesis, Baker et al. (2003) estimated correlations between accumulated 
engagement in several different types of structured practice activities — 
“Watching games on television,” “Weight training,’ and “Organized 
training” — and level of competitive performance and found that they ranged 
from —0.51 to 0.62. Similar evidence for differential effects of practice 
activities was found in Young’s (1998) dissertation, where the total sum of all 
12 tracked activities had a non-significant correlation (r = 0.12) with 
objective performance for middle-distance runners. In the published 
reanalysis of the males in this study (the vast majority of participants), Young 
and Salmela (2010) found that accumulated estimates of several specific 
training activities, such as weight training for power, significantly 
differentiated national level, regional level, and club level male runners. 
There are a small number of studies that have been able to identify 
natural and experimental variations in practice behavior that are significantly 
associated with increased objective performance among expert performers. 
For example, in a pioneering study McKinney and Davis (2003) analyzed the 
outcome of emergency situations during flying by airforce pilots. They found 
that if prior to the emergency event the experienced pilots had practiced the 
Same emergency situation during supervised training in the simulator, they 
were reliably more successful in dealing with the actual event. There is also 
an example of improving endurance of soccer performance by training 
aerobic fitness. Helgerud, Engen, Wisloff, and Hoff (2001) conducted an 


experimental study with randomly assigned groups of international junior 


level soccer players. They demonstrated that additional high-intensity 
training for eight weeks was associated with increases in objective 


performance during soccer matches. 


Changes in the Effects of Practice Alone at Different Ages, Skills, and 
Stages of Careers 
Even if a student is being taught by a music teacher, it does not guarantee that 
all their practice alone meets the criteria for deliberate practice, such as 
working on a Clear goal assigned by their teacher and using the recommended 
methods of practice. Video analyses of young music students show that 


beginners often do not correct their errors, but rather simply repeat their same 


performance (McPherson & Renwick, 2001). Consistent with this finding, the 
amount of practice alone is not correlated with performance for beginning 


music students (Lehmann, 1997). In order for individuals to be able to sustain 


concentration, evaluate their performance attempts, and make appropriate 
adjustments for subsequent attempts, they need to have developed the 
necessary representations (see Figure 38.4) and be very motivated to use their 
practice time effectively. Ericsson et al. (1993) argued that these conditions 
were met for advanced music students at a music academy with the 
commitment to become professional musicians. When these students practice 
alone, it is reasonable to assume that their practice is focused on improving 
specific aspects of performance (cf. deliberate practice). 

The concentration by professionals is nicely illustrated in a study of 
singers by Grape, Sandgren, Hansson, Ericsson, and Theorell (2003). 
Whereas the amateur singers experienced the lesson as self-actualization and 


an enjoyable release of tension, the professional singers increased their 


concentration and focused on improving their performance during the lesson. 
Even for the most accomplished experts, there are limits on daily durations of 
full concentration, and Ericsson et al. (1993) found a maximum of four to 
five hours of work per day for musicians, authors, and scientists and an 
associated need for rest, naps, and sleep. 

When music students graduate from a music academy and accept jobs as 
professional musicians, the time dedicated to practice alone is reduced 
(Krampe & Ericsson, 1996). More importantly, the practice alone time 
involves learning new music pieces that may have limited relevance for 
improving music skill. Furthermore, practice alone for professional musicians 
is also concerned with maintaining an already attained music performance. 
For example, when Krampe and Ericsson (1996) tested expert pianists 
ranging in age from 50 to 70 years old with musically relevant tasks, the 
older expert pianists’ performance was correlated with the amount of practice 
alone. In fact, older expert pianists, who maintained practice alone for around 
10 hours per week could not be distinguished from young expert pianists 
practicing for around 30 hours per week (see Krampe & Charness, Chapter 
42, this volume, for the effects of practice in older experts). It is important to 
note that the practice activities of older expert pianists differ qualitatively 
from those of younger expert pianists. A larger proportion of the older expert 
pianists’ practice was spent with the goal of simply maintaining their 
technical skill (maintenance practice). In contrast, younger expert pianists 
spent much more time on deliberate practice, where they improved a 
particular aspect of the performance with training tasks recommended by 


their teacher. 


Problems Quantifying the Amount of Accumulated Practice and 
Assessing Its Relation to Attained Performance 
If continued engagement in a particular type of practice activity improves 
performance, one would predict that individuals who have spent more time 
engaging in that type of practice would have attained a higher level of 
performance — at least on the average. If one assumes that all practice 
activities improve performance then one could simply sum up all hours of 
any type of practice engaged in by a given individual in a given domain. In 
that case one would then only have to ask individuals to estimate how much 
time they have spent with a given activity per week across every year of 
engagement, whether it involved group practice, lectures, scrimmages, or 
competitions. This approach to quantifying amount of practice and its relation 
to attained performance is completely inconsistent with the expert 
performance framework and its claim that the effects of different types of 
practice activities on increased performance differ depending on the type of 
practice and the needs of a given trainee. As shown in this chapter there is a 
lot of evidence that the duration of some types of common practice activities 
such as playing games is not associated with optimal improvement; often the 
benefit is not statistically reliable. The most effective practice would be 
deliberate practice, where trainees work on an individualized set of practice 
goals with practice activities designed by a teacher or coach as illustrated in 
Figure 38.3. Only when a teacher or coach designs the individualized practice 
on a daily or weekly basis is it appropriate to assume that the type of practice 
engaged in by a trainee over time is optimal and thus every hour should be as 


effective if the performer conducts training with full concentration and effort. 


When the durations of many different types of domain-related practice 
activities are simply added together to form a single estimate of all 
accumulated practice in a given domain, these estimates would not be able to 
quantify how much practice could maximally predict attained performance. 
In fact, Macnamara and Hambrick and their colleagues (Macnamara, 
Hambrick, & Oswald, 2014; Macnamara, Moreau, & Hambrick, 2016) have 
conducted meta-analyses to estimate the relation between performance 
indicators and sums of domain-specific practice activities accumulated during 
individuals’ entire career in the domain. These meta-analyses conclude that 
the total number of hours of practice is significantly related to an attained 
level of performance, but cannot account for all of the variance in 
performance. A careful examination (Ericsson, 2016) of the datasets analyzed 
by Macnamara et al. (2016) showed that many estimates of accumulated 
hours of different types of practice in sports are summed up across a wide 
range of different practice activities, such as team training, scrimmages, 
playing games in competition, and even “watching games on television.” 
Given that different practice activities contribute differentially to 
improvement, as discussed earlier in this chapter, adding all hours of activity 
into a sum will not estimate the upper bounds on effects of practice. Most 
importantly, Macnamara et al.’s (2016) estimates did not restrict their 
analysis to Ericsson et al.’s (1993) original definition of deliberate practice, 
where athletes would be expected to engage in individualized training with a 
coach, in personalized training tasks with explicit goals, immediate feedback, 
and opportunities for repetition until mastery (Ericsson, 2016). Similar 
criticisms have been addressed to Macnamara et al.’s (2014) meta-analysis 
and argue that their findings do not provide valid information about the upper 


bounds of the effects on performance attributable to practice (Ericsson, 


2014c, 2014d; Platz, Kopiez, Lehmann, & Wolf, 2014). Instead of summing 
up all hours of practice to quantify the effects of practice by a single number 
and correlating it with an index of attained performance, the expert 
performance approach involves searching for ways to study the effects of 
particular practice activities on detailed aspects of performance, and the 
associated refined mechanisms. 

A first step toward making progress in understanding the factors 
influencing the attainment of expert performance would be to use a shared 
vocabulary. Many of the studies refer to deliberate practice and even 
reference Ericsson et al. (1993), but then use different definitions of 
deliberate practice, which include individualized practice activities that are 
not guided and designed by a teacher or even in some cases practice that 
doesn’t involve individualized training with immediate feedback (most 


notably, Campitelli & Gobet, 2008; Gobet & Charness, Chapter 31, this 


volume; Howard, 2013; Macnamara et al., 2014, 2016). Future research 
should use more refined descriptions of specific practice activities with 
measurement of the resulting particular changes in measurable performance. 
Similarly, other factors involving individual differences in genetic 
endowment should be described in terms of which gene or genes, and when 
and how these genes are activated and change cells in the body, which 
ultimately generates individual differences in expert performance. Ericsson 
(2014a) reviewed the available evidence for the influence of individual genes 
and heritability inferred from twin studies and found no robust evidence for 
genetic influences on attaining expert levels of performance, beyond the well- 
known innate effects due to height and body size. This of course does not 
mean that future research will not be able to uncover unique genes or 


combinations of genes that are predictive of expert performance in various 


domains. Until such genes are rigorously identified, however, it will not be 
possible to specify how the absence of these gene combinations limits the 
attainable performance of individuals. Unattainability will always be 
contingent on knowledge about current training methods and thus might not 
be true for new and different practice techniques and training histories. In the 
foreseeable future our knowledge of specific factors that are correlated with 
attained level of performance will grow incrementally with the development 
of improved knowledge about types, amounts, and timing of effective 
training as well as knowledge about potentially relevant genes. Given this 
growing knowledge it would be premature to make claims about how much 
variance of individual differences in expert performance could never be 
accounted for by practice or unique gene combinations. 

It is important to realize that significant correlations between the amount 
of particular types of training and improved performance do not by 
themselves imply that practice must cause the observed improvements in 
performance. A commonly held view is that the success of talented 
individuals leads them to practice more, and the failure of the less talented 
leads them to practice less and then eventually drop out (Sternberg, 1996; but 
see Ericsson, 1996 for a discussion). An alternative account within the expert 
performance approach is that reductions in practice take place first and these 
reductions will eventually lead to inferior performance and even dropout. De 
Bruin, Smits, Rikers, and Schmidt (2008) directly evaluated these two 
hypotheses by collecting information on the amount of practice and their 
chess ratings from a group of persisting chess players and players dropping 
out in a sample of young elite players. Most importantly, they found that a 
reduction in weekly serious chess study alone led to a decrease in 


performance and eventually to the dropping out of this elite group. There was 


no evidence that inferior performance led individuals to become discouraged 


and thus reduce their practice. 


Summary 


More generally, within the expert performance framework, it is not possible 
to equate the effects of practice between and within individuals. A teacher or 
a coach will assess a given individual’s current performance and identify 
appropriate training goals and associated practice methods. Thus, what 
constitutes deliberate practice will differ as a function of particular 
individual’s state of skill development. Scores on tests of cognitive ability 
and intelligence are primarily correlated with performance of beginners, and 
the correlations diminish as higher performance is attained through the 
mediation of acquired mechanisms (Ericsson, 2014a). The acquisition of 
mechanisms suggests the likelihood of individual differences in these 
mechanisms and representations, which would influence the effectiveness of 
future practice alone and deliberate practice. Understanding the structure of 
acquired mechanisms and their associated cognitive processes would require 
elicitation of think-aloud protocols and other types of verbal reports during 
practice sessions (see Ericsson, Chapter 12, this volume). When the 
achievement of expertise is conceptualized as a sequence of attained aspects 
(states in Figure 38.3), it becomes clear that the paths toward expert 
performance as well as the effects of the duration of particular training 
activities might differ among individual performers in the same domain of 


expertise. 


General Remarks on Attaining Expert 
Performance as a Sequence of States 


According to the expert performance framework it is clear that our 
knowledge of the acquisition of expert performance in a particular domain of 
activity is quite limited until the sequences of attained states of improved 
performance for individual performers can be identified and measured. In 
many domains, the successful capture of expert performance has allowed us 
to identify mechanisms mediating the superior performance on representative 
tasks. In another chapter in this handbook (Chapter 12) I describe the process 
of capturing expert performance and collecting think-aloud protocols and 
show how these mechanisms mediating superior performance are refined 
though individualized deliberate practice. A critical method for identifying 
the causal influence of the mediating mechanisms is to interfere with them 
experimentally. 

In Chapter 36 of this handbook I describe the acquired memory skills 
that support increased working memory capacity during expert performance. 
The present chapter focuses on evidence for some of these mechanisms and 
how they were attained through development to adulthood, including the 
influence of innate factors, such as height and body size. Most of the 
emphasis has been given to how mechanisms mediating the superior 
performance are acquired as the result of engagement in practice, especially 
deliberate practice as well as individualized practice with feedback, but 
without a teacher. It is obvious that attaining expert performance requires 


multiple kinds of engagement in domain-related activities beyond 


individualized practice, such as travel to and from training and competitions, 
relaxation and rest, engagement in group practice for teams, orchestras, and 
so on. The hypotheses proposed by the expert performance framework is that 
many activities may be important and necessary, but when it comes to 
explaining individual differences in performance the amount and quality of 
engagement in some particular types of practice may be particularly 
important. 

In this chapter I have reviewed evidence that deliberate practice has 
been found to be an effective way to change (improve) performance, but this 
type of practice seems to be rare among amateurs and beginners. This type of 
practice requires a teacher who can assign attainable performance goals for 
changes, recommend available individualized practice techniques with 
immediate feedback, and review attained performance improvements. Even 
among skilled performers, many types of practice activities have been shown 
to violate one or more of the requirements for deliberate practice. Many types 
of practice are not individualized and adapted to the needs of each performer, 
such as practice activities involving a group or team. Most practice activities 
are not performed individually where the performer can get immediate 
feedback about their performance and opportunities to gradually improve the 
same or similar performance. Deliberate practice is just one type of practice 
and this type has been discovered to yield impressive improvement of 
performance and it combines several elements, such as effective teachers, 
immediate feedback, and opportunity for repetitions. When individuals 
engage in another type of practice that has one or more of these elements, we 
would predict some benefit for improving performance. It is also important to 
realize that engagement in competition as well as training and performing in 


teams has unique value for developing superior competitive performance. 


Subsequent analysis of videos of competitions performed by individuals and 
teams permits performers and coaches to identify aspects in need of targeted 
improvement. Competitions also provide opportunities for individuals to 
incorporate and integrate improvements attained during deliberate practice 
when performing under representative competitive conditions in the domain. 
Although we are already gaining understanding about how performers 
improve with deliberate practice and reach expert levels, it is unlikely that we 
will ever be able to fully understand and predict future innovations in 
technique and training methods. We may be able to reproduce the path of 
development that elite performers have taken to reach their highest levels of 
performance in the past. We may also be able to help performers in one 
domain of expertise, such as laparoscopic (pinhole) surgery, learn about the 
best training methods that have been developed in domains with a longer 
tradition, such as violin performance. We may even be able to work in 
collaboration with world-class performers who are working on improving 
their performance to new and undiscovered heights. At the highest levels of 
expert performance, the drive for improvement will always involve search 
and experimentation at the threshold of understanding, even for the masters 


dedicated to redefining the meaning of excellence in their fields. 
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An Ecological View of Expertise 


For many, the word “expertise” calls to mind highly skilled individuals and 
their accomplishments in athletics, the arts, and the professions. Experts are 
widely recognized for devoting years of effort to honing their craft, 
consistently demonstrating advanced performance, and achieving elite status 
in a given domain (Feltovich, Prietula, & Ericsson, Chapter 6, this volume). 
Experts also foster community development by serving as models for, and 


active trainers of, aspiring juniors (Billett, 2006; Collins, 2013; Collins & 


Evans, Chapter 2, this volume; Mieg & Evetts, Chapter 9, this volume). 
Scholarly interest in expertise, which spans psychology, sociology, and 
philosophy, appears driven by three questions regarding its nature and 
attainment (Feist, 2013; Papierno, Ceci, Makel, & Williams, 2005): (1) What 


talents, traits, or attributes enable advanced skill acquisition?; (2) What 


conditions, activities, and experiences reliably produce experts?; and (3) How 
does the interaction of individual characteristics and developmental 
opportunities give rise to expertise? Although it is generally accepted that 
expert performance reflects the joint contribution of initial conditions and 
experience, perspectives differ on the nature of initial conditions (e.g. 
genetics, non-inherited characteristics, or both), the impact of practice, and 
the role of domain characteristics and societal factors in defining experts and 


expertise (Chi, 2006; Collins, 2013; Feist, 2013; Feltovich et al., Chapter 6, 


this volume; Mieg & Evetts, Chapter 9, this volume; Papierno et al., 2005). 


In this chapter, we propose an integrative conception of expertise that 
treats basic human abilities as rudimentary, developing forms of expertise 
(Sternberg, 1998a) and that attempts to account for everyday expert 
performance in a range of settings, including school and work (see also 


Collins, 2013; Collins & Evans, Chapter 2, this volume). From this 


perspective, we ask: What particular match of abilities and environmental 
conditions enables success in everyday life? How is this match attained? (See 
also Feist, 2013; Papierno et al., 2005; Shanteau, 1992.) We argue that 
everyday experts are capable of achieving the right fit between their abilities 
and the demands of a situation, and that the development of expert 
performance, whether demonstrated in such everyday feats as reading and 
writing or in exceptional accomplishment, reflects the outcome of people’s 
engagement in the world around them. 

Toward this end, we summarize a program of psychological research 
exploring expertise based on the theory of tacit knowledge and practical 
intelligence (Stemberg, 1988, 1997; Wagner & Sternberg, 1985). This 
research seeks to explain the cognitive mechanisms involved in people’s 
general ability to adapt to, select, and shape their environment in the pursuit 
of everyday expert performance. We begin by _ presenting our 
conceptualization of expertise as a confluence of personal capability and 
environmental opportunity, highlighting the convergence of our perspective 
with ecological approaches to understanding human development and 
behavior. We then describe the role that practical intelligence and tacit 
knowledge play in the acquisition of advanced skill, noting similarities to 
ecological perspectives. We then describe the measurement and development 


of expertise using approaches based on practical intelligence and tacit 


knowledge theory and related ecological theories. Finally, we suggest 


directions for future research. 


An Ecological View of Expertise 


Consistent with cognitive conceptions of expertise (Feltovich et al., Chapter 


6, this volume; Sternberg, 1994), an ecological view of expertise maintains 


that expertise is domain-specific and that the attributes of experts may be 
specific to a time and place. This view maintains the existence of 
foundational information-processing capabilities, while acknowledging that 
the requisite knowledge and skills for expert performance are defined quite 
differently depending on the environment in which people develop (e.g. Feist, 
2013; Horn & Masunaga, 2006). Importantly, an ecological view of expertise 
recognizes the diversity of capabilities that comprise expert performance and 
posits that expertise exists in degrees reflecting the amount of symmetry 
between a person’s set of capabilities and the demands of the present 


situation (see also Loevinger, 1966; Wittman & Siife, 1999). 


Conceptualizing expertise as the confluence of general information 
processing capabilities, knowledge depth and _ organization, and 
environmental opportunity requires understanding how general psychological 
mechanisms interact with acquired knowledge and situational conditions to 
produce expert performance. The theory of practical intelligence and tacit 
knowledge (Sternberg, 1988, 1997; Wagner & Sternberg, 1985) explicitly 
addresses the interchange between information processing capability and 
experience in particular contexts, making it a useful source of insight into 
expertise. Specifically, the theory posits that the development and application 
of tacit knowledge, a critical aspect of success in everyday life, occurs 


through a practically intelligent cycle of inquiry, knowing, and action, that is, 


a cycle of engaging the environment, acquiring tacit knowledge, and using it 
to perform (see Cianciolo, Antonakis, & Sternberg, 2004). 

The theory of practical intelligence and tacit knowledge is consistent 
with conceptualizations of everyday expertise typical of ecological 
approaches to understanding human development and behavior (Cianciolo et 
al., 2004). Central to these approaches is taking person—environment 
coupling as the unit of analysis, exploring the nature and improvement of 
person—environment fit, and recognizing the active role of the human in 
attaining this fit (Gibson & Pick, 2000; Greeno, 1994; Kirlik, 2006). On this 
view, expertise develops through the reciprocal interaction of maturation and 
action in the pursuit of functionally relevant goals (Gibson & Pick, 2000). 
Ecological notions of expertise originated in studies of human perception 


(e.g. Gibson, 1979; Tolman & Brunswik, 1935) and have since been adopted 


to account for everyday expert performance in basic human ability, judgment 
and decision making, and socio-technological systems (e.g. Brehmer & 
Joyce, 1988; Gibson & Pick, 2000; Kirlik, 2006; Wittman & Siif$, 1999). 


The value of taking an ecological perspective on expertise is that it 


provides an opportunity to rethink such fundamental issues as how expertise 


is defined, particularly how to expand the role of context in this definition 


(Collins, 2013; Mieg & Evetts, Chapter 9, this volume). We focus on the 
theory of practical intelligence and tacit knowledge here as an instance of the 
ecological perspective because it was developed as an explicit attempt to 
account for everyday expertise from a cognitive perspective that is aligned 
with, yet more expansive than, predominantly psychological notions of 
expertise (Feltovich et al., Chapter 6, this volume; Sternberg, 1994). 
Highlighting ties between the theory of practical intelligence and tacit 


knowledge and the broader family of ecological perspectives opens up new 


avenues of inquiry and promotes the integration of psychological theories of 
expertise with perspectives from other disciplines (e.g. Billett, Harteis, & 
Gruber, Chapter 8, this volume; Collins, 2013; Collins & Evans, Chapter 2, 


this volume; Feist, 2013; Mieg & Evetts, Chapter 9, this volume; Greeno, 
1994). 


Tacit Knowledge and Practical Intelligence as 
Expertise 


Tacit Knowledge: Domain-Specific Actionable Cognition 


The word “tacit” is used to characterize actions carried out without the use of 
words or speech and to describe common arrangements that have arisen 
without explicit agreement or discussion (Oxford English Dictionary, 2015). 
Tacit knowing therefore reflects a person’s agency in a given context, 
enabled by implicit interaction with the environment. Although the idea that 
people’s actions are subject to subconscious influences dates back to 
Sigmund Freud, Michael Polanyi (1958, 1966) was among the first to 
propose formally the concept of tacit knowing, noting its influence on 
perception and scientific thinking. Polanyi (1966) argued that “we can know 
more than we can tell” (p. 4) and that tacit knowing is a foundational resource 
for a wide range of complex skills, from tool use to application of the 
scientific method. 

Polanyi (1958, 1966) claimed, for example, that when humans use a tool 
they are unaware of how the action of the tool corresponds to the sensations 
the tool is producing on their palm; such personal knowledge functions on a 
tacit level, enabling people to attend solely to the actions of the tool. The tool 
becomes an extension of the person such that, in using the tool to achieve a 
particular aim, the person cannot articulate how she uses the tool any more 
than she can articulate how she uses her own hand. Polanyi (1958) also 
emphasized the experiential nature of tacit knowing — that it must be acquired 
by example and practice, often implicitly. Ecological psychologists Gibson 
and Pick (2000) have similarly noted the experiential nature of human 
development, highlighting its functional aims. For example, they argued that 
tool use is acquired in the service of accomplishing a desired goal, such that 


what is immediately accessible to the learner is not awareness of how to 


manipulate the tool but rather what ends the tool can be used to accomplish 
when manipulated in various ways. 

Since Polanyi’s work, scholars from domains as diverse as linguistics 
(Dahl, 2000), cognitive psychology (Reber, 1989; Reber & Lewis, 1977), 
differential psychology (Wagner & Sternberg, 1985), medicine (Engel, 2008; 


Henry, 2006), social science (Collins, 2013), organizational management 
(Nonaka & Takeuchi, 1995; Venkitachalam & Busch, 2012), software 
development (Heredia, Garcia-Guzman, Amescua, & Sanchez-Segura, 2013; 
Ryan & O’Connor, 2009), and even bioterrorism (Revill & Jefferson, 2014) 


have independently worked toward understanding the nature and acquisition 


of tacit knowledge. In this chapter, we focus primarily on the work of Robert 
J. Sternberg and Richard K. Wagner and their colleagues, because they have 
explored extensively the nature, measurement, and acquisition of tacit 
knowledge as it relates specifically to an integrative, person—environment 
model of expertise. 

Wagner and Sternberg (1985) defined tacit knowledge as “knowledge 
that usually is not openly expressed or stated ... is not directly taught or 
spoken about, in contrast to knowledge directly taught in classrooms” (pp. 
438-439), with the qualification that “we do not wish to imply that this 
knowledge is inaccessible to conscious awareness, unspeakable, or 
unteachable, but merely that it is not taught directly to most of us” (p. 439). 
Highlighting the essentially functional role of tacit knowledge, Sternberg and 
his colleagues have noted that it is action-oriented and procedural in nature, 
representable as a complex set of condition—action statements associated with 
accomplishing a given goal (see e.g. Sternberg et al., 2000). For example, 
tacit knowledge held by a university faculty member regarding successful 


participation in a committee meeting might be something like: 


IF <the allotted time for the meeting is coming to an end> 
And 


IF <the meeting has been taken up almost entirely by contentious debate 


about a single topic> 
And 
IF <the committee chair makes a closing call for other business> 


THEN <refrain from bringing up a new idea regarding the previously 


debated topic> 


BECAUSE <this prevents irritating other committee members who can 


influence the idea’s reception and ultimate adoption> 


The concept of tacit knowledge resembles the ecological concept of 
attunement to constraint (Greeno, 1994). Constraints comprise the general 
person—environment interaction necessary for successful performance of a 


given mental or physical task (Greeno, 1994). Using the above example, the 


<action>—<outcome> constraint would be <timing of 
contributions>—<receptive colleagues>, and it would only apply in 
environments where one is discussing ideas with colleagues. These 
environments could be quite varied, however, to include face-to-face 
meetings, online meetings, or even email exchanges and any combination of 
different colleagues. Being attuned to constraints enables consistent goal 
accomplishment despite surface-level variation in situations because a 
multiplicity of specific actions can satisfy a constraint. One’s store of tacit 


knowledge contains this multiplicity of condition—action—outcome 


contingencies and enables consistently high performance in a range of 
situations. 

Tacit knowledge also shares similarities with the construct of procedural 
knowledge — “how to” knowledge that is automatized with practice, such as 
typing (Ackerman, 1988; Anderson, 1982; Fitts & Posner, 1967). Both 
procedural knowledge and tacit knowledge are action-oriented, gained from 
experience, applied subconsciously, and often difficult to verbalize. 
However, unlike procedural knowledge, tacit knowledge is not an automatic 
response produced from repeated exposures to the same patterns of stimuli. 
Rather, it is an adaptive intellectual resource that can be applied to novel 
situations because it stems from multimodal and reciprocal interaction 
between people and their environment. That is, as a person’s capability 
expands, so too do her opportunities for action and further development, as 
noted by Gibson and Pick (2000, p. 187): “Repeated experiences of an event 
with a comparable functional relation to oneself and a comparable outcome, 
over varying details of context ... provide a rich basis for developing 
meaningful knowledge” (see also Karmiloff-Smith, 1992; Schwartz & 
Bransford, 1998). In this way, tacit knowledge goes beyond procedural 
knowledge in accounting for the domain-specific, yet consistently successful 
application of complex cognitive processes that characterize expert 
performance. 

Tacit knowledge may also seem similar to job knowledge (Schmidt & 
Hunter, 1993), but there are important conceptual differences here too. Job 
knowledge typically is conceptualized as declarative knowledge, such as facts 
and concepts required for successful job performance, and commonly is 


taught explicitly (e.g. Hunter, 1983; Ree, Caretta, & Teachout, 1995). 


However, expert occupational performance requires more _ than 


implementation of facts and concepts learned via formal instruction; 
problem-solving and anticipatory thinking also are needed (see, e.g. Braude, 
2009; DuBois & Shalin, 1995; Henry, 2006). Learned on the job, tacit 


knowledge facilitates expert occupational performance by bridging the gap 


between formally educated knowledge and operational experience (see e.g. 
Braude, 2009; Henry, 2006; Sternberg & Horvath, 1999; see also Billett et 
al., Chapter 8, this volume). In different domains, using different measures, 
Tan and Libby (1997) and Ryan and O’Connor (2009) found weak 


correlations (0.19—0.22) between tacit knowledge and technical knowledge, 


suggesting that although tacit knowledge is an aspect of job knowledge, the 
two constructs are not interchangeable. In addition, tacit knowledge need not 
be job-related, but may facilitate expert performance in non-work domains as 
well (Cianciolo et al., 2006; Elton, 2010; Insch, McIntyre, & Dawley, 2008; 
Sternberg & Rainbow Project Collaborators, 2005). 


Practical Intelligence: Acquisition and Use of Tacit Knowledge 


Practical intelligence is defined as the ability to acquire tacit knowledge from 
experience and to apply this knowledge to handling practical problems in 
which the information necessary to determine a solution strategy is often 


incomplete (see e.g. Cianciolo et al., 2004; Sternberg et al., 2000). Following 


Neisser’s (1976) distinction between academic and naturalistic intelligence, 
Wagner and Sternberg (1985) posited practical intelligence as an explanation 
for the less-than-perfect correlation between tests of general intelligence or 
academic knowledge and everyday practical achievements, including job 
performance. Sole reliance on a unitary construct and/or single measurement 


method to account for expertise necessarily defines it more narrowly than 


actual expert performance, which is multifaceted (Loevinger, 1966; Wittman 


& Stiff, 1999). The reciprocal relationship between practical intelligence and 


tacit knowledge reflects Sternberg’s (1998a) conceptualization of human 


ability as a form of developing expertise in that the acquisition and 
application of tacit knowledge — foundational to successful goal-directed 
action — relies on general cognitive processes but is domain-specific and 
improves with experience. Henry (2006) independently noted the central role 
of tacit knowledge in practically intelligent behavior when he observed that 
“problem solving requires imaginative groping towards a solution guided by 
tacit clues, since tacit knowledge is far more extensive than and always 
precedes explicit knowledge” (p. 190). 

Sternberg (1988) posited that tacit knowledge acquisition is driven by 
the cognitive processes involved in the often subconscious manipulation of 


information encountered in novel situations in order to learn from experience. 


These processes are (a) selective encoding; (b) selective combination; and (c) 
selective comparison. Selective encoding is the selection of information from 
the environment that is relevant to understanding the current situation or to 
solving the problem at hand. Selective combination is the integration of 
multiple pieces of selectively encoded information into a unified whole that 
creates a meaningful pattern and, eventually, a knowledge structure. Selective 
comparison is the comparison of newly formed patterns of information or 
knowledge structures to previously formed ones. Accurate selective 
encoding, selective combination, and selective comparison result in an 
increased store of tacit knowledge and, consequently, more practically 
intelligent behavior. 

Practical intelligence, defined as the ability to acquire and use tacit 
knowledge, resembles the “information pickup” and perceptual learning 
activities described by Gibson and Pick (2000) as “the very foundation of 
intelligence” (p. 21). Information pickup is accomplished via a combination 
of exploratory and performatory activities, which serve to yield and confirm 
knowledge, respectively. Exploratory activities yield knowledge by gathering 
information about the world (e.g. fumbling about in the dark to find a light 
switch). Performatory activities are undertaken to produce a particular, 
expected result (e.g. flipping a switch known to operate a particular light), 
thereby validating one’s understanding of how the world “works.” These 
processes are not limited to the development of perceptual skill but also are 
applicable to conceptual development and higher forms of human agency 


(Gibson & Pick, 2000). A related concept is that of “epistemic action” (Kirsh 


& Maglio, 1994) in which a person creates new information when 
environmental cues are insufficient to ascertain a state of affairs (Kirlik, 


1998). For example, a physician may request laboratory tests when physical 


examination does not permit confident diagnosis. Epistemic action is a mode 
of exploratory activity that reflects an advanced form of selective encoding. 
Framed as a general ability, the construct of practical intelligence may 
have small overlap with the well-established construct of general intelligence. 
Although the exact nature of general intelligence is yet unknown, it is widely 
recognized as the highly general capability to process information, and it is 
believed to have specific neurological substrates (e.g. Duncan et al., 2000). 
To the extent that neurological functioning undergirds all mental activity, 
practical intelligence should show some relation to general intelligence, and 
both constructs should relate to expert performance (see Hill & Schneider, 
2006). However, general intelligence is viewed as a relatively stable 
characteristic, making it theoretically distinct from practical intelligence, 


which is viewed as modifiable with experience (Sternberg, 1998a). Practical 


intelligence more closely resembles crystallized intelligence through their 
shared experiential aspects and association with knowledge acquisition. 
Crystallized intelligence has been defined as the outcome of “experiential- 


educative-acculturation influences” (Horn & Cattell, 1966, p. 254) on one’s 


biological capacity, is declarative in nature, and commonly is assessed using 
tests of basic vocabulary, reading comprehension, and other verbal skills. 
Practical intelligence, however, acts on and develops out of a much broader 
range of experiences than crystallized intelligence — a process that occurs 
well into adulthood — and the more procedural, domain-specific knowledge 
acquired is better suited to everyday expert performance (see also Horn & 
Masunaga, 2006). 

In general, tacit knowledge is weakly correlated with general 
intelligence (r <0.20; Cianciolo et al., 2004, Legree, Heffner, Psotka, Martin, 
& Medsker, 2003; Tan & Libby, 1997; Wagner, 1987; Wagner & Sternberg, 


1985, 1990; though see Colonia-Wilner, 1998 and Matthew & Sternberg, 


2009 for slightly higher correlations). This weak correlation does not appear 


due to the domain-specificity of tacit knowledge; where relatively domain- 
general assessments of tacit knowledge have been used, low correlations with 
general and crystallized intelligence still have been found (r = 0.03 to 0.19, 
Cianciolo et al., 2006). In addition, Cianciolo et al. (2006) found that a 
higher-order latent factor of practical intelligence, which reflected the 
commonality among measures of general tacit knowledge, correlated 0.34 
with fluid intelligence and 0.20 with crystallized intelligence. 

Because practical intelligence is theorized to develop through effortful 
engagement with the environment, it also could be argued that acquiring tacit 
knowledge and acting in a more practically intelligent fashion depend on 
multiple non-ability characteristics that relate to learning and performance, 
such as personality and motivation (McDaniel & Whetzel, 2005). With some 
reflection, this argument could be made for the other psychological constructs 
listed above, including general intelligence as it is typically measured 
(Cianciolo & Sternberg, 2004). Practical intelligence is theorized to be 
distinct from non-ability characteristics because it comprises a person’s use 
of his experiences to accomplish functionally relevant goals and not, in 
particular, his social nature or drive to succeed. Tacit knowledge has shown a 
weak association with several aspects of personality, including sociability, 
social presence, self-control, and achievement via conformity (r = 0.14, 0.29, 
0.19, and —0.05, respectively; Wagner & Sternberg, 1990). Potential 
conceptual similarities between practical intelligence and learning styles 


(Kolb & Fry, 1975), which posit that cognitive style and environment interact 


to promote experiential learning, warrant further investigation. Armstrong 


and Mahmud (2008) have shown, for example, that having a congruent 


learning style and work setting was associated with greater tacit knowledge 
and occupational success among Malaysian public sector employees (see also 
Armstrong & Li, 2017). 


Assessing Expertise from an Ecological 
Perspective 


Measuring expertise from an ecological perspective requires that the behavior 
sampled reflect the outcome of people engaging with their environment and 
acquiring personalized, actionable knowledge from their experience. 
Sampling multiple kinds of behaviors is needed in order to triangulate on this 


construct via converging operations (Garner, Hake, & Eriksen, 1956; 


Loevinger, 1966). The methods Sternberg and his colleagues have used to 
assess tacit knowledge have ranged from brief, low-fidelity simulations to in- 
depth case study scenarios. They also have differed in the degree to which 
they assess specialized tacit knowledge, ranging from quite specific, 
including business management (Wagner & Sternberg, 1991), sales (Wagner 
& Sternberg, 1989), and military leadership (Hedlund et al., 1998; Hedlund, 
Wilt, Nebel, Ashford, & Sternberg, 2006), to more general, including 


everyday living and entry-level business skills (Chart, Grigorenko, & 
Sternberg, 2008; Cianciolo et al., 2006; Sternberg & Rainbow Project 


Collaborators, 2005). Other scholars have developed their own tacit 
knowledge assessments in areas as diverse aS academia (Insch et al., 2008; 
Nestor-Baker & Hoy, 2001; Somech & Bogler, 1999), auditing (Tan & 
Libby, 1997), and even driving (Legree et al., 2003), to include measures of 


team tacit knowledge (Ryan & O’Connor, 2009). Other interesting measures 


have been used by scholars taking an ecological or practical approach to 


operationalizing expert performance (Charlin, Roy, Brailovsky, Goulet, & 


van der Vleuten, 2000; Kirlik, 2013). We discuss these different forms of 


measurement below. 


Tacit Knowledge Inventories 


Tacit knowledge inventories use a situational-judgment testing (McDaniel & 
Nguyen, 2001; McDaniel & Whetzel, 2005) or low-fidelity simulation 
(Motowidlo, Dunnette, & Carter, 1990) format, typically to assess highly 


domain-specific tacit knowledge. Examinees completing a tacit knowledge 
inventory are presented with a series of brief vignettes, each of which depicts 
a practical problem that must be solved and provides a set of solution 
alternatives. Examinees must rate each of the solution alternatives for its 
perceived effectiveness or quality using a Likert scale, usually ranging from 1 


(very bad) to 7 (very good). Scores on tacit knowledge inventories are 


derived in a variety of ways (see Sternberg et al., 2000), each of which 
represents the degree to which an individual’s ratings correspond to the 
average ratings of some comparison sample, usually a group of experts but 


sometimes the consensus of a sample from the inventory’s target population. 


Script Concordance Tests 


Developed to assess tacit knowledge in clinical medicine (e.g. Henry, 2006), 


script concordance tests (Charlin et al., 2000) share many similarities with 


tacit knowledge inventories. In these tests, the examinee is provided with a 
brief clinical vignette and is asked to indicate what impact on her clinical 
decision making that new information about the situation would have. The 
aspects of clinical decision making assessed by script concordance tests 
include making a diagnosis, choosing an investigative method (e.g. lab 
testing), or prescribing a treatment. Examinees rate the impact of specific 
new information using a scale of —2 to +2 to indicate whether it makes a 
diagnosis more or less probable or a diagnostic test or treatment more or less 
useful. As with tacit knowledge inventories, stronger correspondence to the 
opinions of a comparison sample — in this case expert physicians — is taken to 
reflect greater tacit knowledge; however, concerns recently raised about 
scoring script concordance tests (Lineberry, Kreiter, & Bordage, 2013) 
suggest that refinement of this approach is needed to reduce the spurious 
impact of test-taking strategies and address the high level of disagreement 


among experts comprising the comparison sample. 


Case Study Scenarios 


Case study scenarios (Cianciolo et al., 2004; Matthew & Sternberg, 2009) 


also provide problem-solving information sequentially over time; however, 
this assessment method involves detailed, in-depth cases and assesses 
practical problem-solving using open-ended questions. Case study scenarios 
simulate the complexity of actual problems by including multiple issues, 
previous actions taken, and contextual information. They also focus on 
practical intelligence by asking examinees to specify the problem as they 
define it, their problem-solving strategies and anticipated outcomes, and the 
environmental and experiential information on which they based their 
approach (see Cianciolo et al., 2004). This assessment method draws from 
approaches commonly used in managerial assessment and education, 
including in-basket tests and case studies. Relative to tacit knowledge 
inventories and script concordance tests, case study scenarios more closely 
simulate actual practical problem-solving, but they take more time for 
examinees to complete, and scoring methods are somewhat more subjective 


and time-consuming. 


Performance Modeling 


Judgment researchers have long used mathematical models to characterize 
humans’ apprehension of environmental conditions (Brehmer & Joyce, 1988; 
Hammond, Hursch, & Todd, 1964). This approach, based on Egon 
Brunswik’s theory of probabilistic functionalism, assumes that people use 
proximal cues (e.g. headache, auras, nausea) to infer the state of a distal 


world (e.g. migraine) (Goldstein, 2006). Regression equations are used to 


depict, on the one hand, the validity of various cues representing a given state 
of the world and, on the other hand, a person’s utilization of those cues to 
make inferences. Conceptualized as the adaptive use of environmental cues, 
expert tacit knowledge can be identified and measured using these 
sophisticated performance-modeling techniques and embedded into complex, 
dynamic tasks, such as command and control simulations (see e.g. Kirlik, 
2013). An advantage of this approach is that creating measures of tacit 
knowledge does not depend on experts’ verbalizations of what they know, 
which can paint a misleading picture of attunement and expertise (Wilson & 
Dunn, 2004). However, the success of applying policy-capturing models to 
represent expertise does appear to depend on the characteristics of the 


judgment task (Shanteau, 1992). That is, judgment tasks with irreducible 


uncertainty (e.g. variable stimuli representing the state of the world, few 
reliable proximal cues, limited outcome information and feedback) tend to 
show fewer differences between expert and novice knowledge (Wigton, 
1996; see also Steurer, Held, & Miettinen, 2013). 


Practical Intelligence, Tacit Knowledge, and 
Expertise 


Tacit knowledge has shown an association with expert performance in 
several diverse domains. For example, Wagner (1987) found among 90 
academic psychologists a correlation of 0.44 between tacit knowledge for 
psychology and number of citations reported in the Social Sciences Citation 
Index in 1982 and 1983. Tacit knowledge also was greater in psychology 
departments ranked higher for the scholarly quality of their faculty (r = 0.48). 
Wagner (1987) also found that tacit knowledge increased with level of 
professional development. That is, he found a significant linear trend in 
scores on a tacit knowledge inventory for academic psychology, with the 
highest scores (indicating greatest deviation from independent expert 
consensus) earned by undergraduate students, intermediate scores earned by 
graduate students, and lowest scores earned by faculty. Wagner (1987) found 
this same trend in a sample of managers, business students, and 
undergraduates. 

In another study of business managers, Wagner and Sternberg (1985) 
found that scores on an early version of the Tacit Knowledge Inventory for 
Managers (TKIM, Wagner & Sternberg, 1991) correlated 0.46 with salary 


and 0.34 with the level of the company where the managers were employed 


(i.e. ranking on the Fortune 500 list). They found in another sample of bank 
managers that tacit knowledge was associated with a variety of indicators of 
success; tacit knowledge correlated 0.48 with percentage of salary increases, 


0.56 with ratings of success in generating new business, 0.29 with ratings of 


personnel management capability, and 0.39 with ratings of ability to 
implement company policy. Wagner and Sternberg (1990) also examined the 
correlation between tacit knowledge inventory scores and performance 
ratings on two management simulations. This correlation was 0.61, with tacit 
knowledge showing a stronger association to performance than several 
measures of cognitive ability and personality. 

Tan and Libby (1997) also explored the relationship between tacit 
knowledge and expertise in a business setting. They administered a tacit 
knowledge inventory for auditing based on the TKIM (Wagner & Sternberg, 
1991) to 100 financial auditors at the ranks of staff, senior, manager, and 
partner employed by the Singapore office of a major accounting firm. 
Mirroring Wagner (1987), they found that tacit knowledge became more 
strongly associated with success as the emphasis of job demands shifted away 
from the application of technical skills to the handling of complex practical 
problems, such as competing goals and career management (Tan & Libby, 
1997; see also Colonia-Willner, 1998). The opposite pattern was found for 
technical knowledge. Armstrong and Mahmud (2008) more recently found a 
similar association between tacit knowledge for management and managerial 
work experience among employees in the public sector. 

Exploring military applications, Hedlund et al. (2003) compared 
officers’ scores on the Tacit Knowledge for Military Leadership inventory 
(TKML; Hedlund et al., 1998) with their leadership effectiveness as rated by 
their supervisors, peers, and subordinates. Leaders’ tacit knowledge was 
found to be largely independent of subordinate and peer ratings, but a notable 
correlation (0.48) between tacit knowledge and supervisor ratings was found 
at the battalion level of command. Consistent with Tan and Libby’s (1997) 


findings, lower correlations were found between tacit knowledge and 


supervisor ratings at the lower levels of command. Leadership expertise 
involves not only tacit knowledge, however, but also the wisdom to use that 


tacit knowledge to good effect (Sternberg, 2003a; see also Antonakis, 


Cianciolo, & Sternberg, 2004a, 2004b). It involves integrating the cognitive, 
motivational, and affective aspects of work (Dai & Sternberg, 2004) and 
recognizing talent rather than mere agreement with the leader’s point of view. 
People tend to value most others who think, feel, and value as they do 
(Sternberg, 1987a, 1998b), but expert leadership requires going beyond 
valuing yes-men and -women to valuing those who can bring divergent but 
useful perspectives to the challenges at hand. 

Legree et al. (2003) studied the relationship between tacit knowledge for 
safe driving and accident history among enlisted army personnel. Their Safe 
Speed Knowledge Test featured 14 brief descriptions of driving conditions 
that varied along the dimensions of weather, traffic, or emotional states, 
among others. Participants estimated how much they would adjust their speed 
in each situation. Scores were derived by calculating the absolute difference 
between an individual’s speed adjustments and the average speed adjustments 


of the entire participant sample (see Legree, 1995), then reflecting this 


difference such that larger scores indicated greater tacit knowledge. Tacit 
knowledge scores were strongly associated with driver-at-fault crash rates. 
Specifically, scoring within one standard deviation of the mean was 
associated with a 2.3 times greater likelihood of crash involvement than 
scoring more than one standard deviation above the mean. Individuals 
scoring more than one standard deviation below the mean were five times 
more likely to be involved in a crash. 

A counterintuitive, but consistent finding in medical research is that 


increased experience, which presumably increases tacit knowledge, has a 


small effect on the accuracy of clinical judgment, a hallmark of medical 
expertise (Spengler & Pilipis, 2015). Moreover, mathematical models of the 
tacit knowledge underlying experienced physicians’ clinical judgments do 
not reliably distinguish them from less experienced physicians (Steurer et al., 
2013; Wigton, 1996). One explanation for this finding is that physicians’ 
clinical judgment is not cultivated using a deliberate practice approach 


(Ericsson, 2015; Klamen, 2015; Spengler & Pilipis, 2015). Despite years of 


formal training, direct observation is minimal (Chisholm et al., 2004; Osman, 
Walling, Mitchell, & Alexander, 2015; Williams & Dunnington, 2006) and 


performance feedback, which can improve diagnostic accuracy (Pusic, 


Pecaric, & Boutis, 2011), is commonly absent. Emerging constructs of 


diagnosis present it as a dynamic aid to iterative and collaborative sense- 
making, rather than an end in itself (Ilgen, Eva, & Regehr, 2016). Process 
constructs of diagnosis may underlie increasing interest in capturing trainees’ 


articulations of their thinking (Duming et al., 2012; Ericsson, 2015; Park et 


al., 2013; Williams & Klamen, 2012), which are commonly used by 
residency supervisors to judge competence and entrust clinical autonomy 
(Kennedy, Regehr, Baker, & Lingard, 2008). 

Perhaps where tacit knowledge is most important to expertise, in the 
end, is in wisdom (Karelitz, Jarvin, & Sternberg, 2010; Sternberg, 2009b, 
2013a) and in teaching for wisdom (Sternberg, 2013b; Sternberg, Jarvin, & 


Grigorenko, 2009). Wisdom involves the ethical use of one’s knowledge and 
skills toward a common good by balancing, over the long as well as the short 
term, one’s own, others’, and larger interests (Sternberg, 20035). 
Recognizing that higher education must produce future leaders who will 
make a positive, meaningful, and enduring difference to the world, Sternberg 


(2010a, 2016) has reported on college admissions tests measuring not only 


analytical skills, but also creative, practical, and wisdom-based skills. He and 
his colleagues have found that including measures of practical intelligence 
increases prediction of both academic and extracurricular success while 
substantially decreasing ethnic group differences on the _ predictive 
assessment. Wisdom draws heavily on tacit knowledge; there is little explicit 
knowledge when it comes to wisdom. Although proverbs are an attempt to 
provide such wise knowledge, they often are vague (“All’s well that ends 
well”), contradictory (“Out of sight out of mind”; “Absence makes the heart 
grow fonder’), or simply not applicable to the wide range of difficulties 
facing individuals and the world. Proverbs have been around for a long time 
but seem to do little to prevent smart people from acting in stupid ways 
(Sternberg, 2002). 

A second, related domain in which tacit knowledge is critical to 
expertise is ethics (Gardner, Csikszentmihalyi, & Damon, 2002; Sternberg, 
2010b, 2012a, 2015) and teaching for ethics (Sternberg, 2012b). American 
schools heavily emphasize the development of content knowledge expertise, 


using standardized tests to measure and analyze it (Sternberg, 2009a), but 


they do not measure how to use this knowledge in an ethical way. The 
implication is that society can develop experts in content knowledge who use 
their knowledge to bad ends. Examples of this, such as the jiggering of 
pollution tests by Volkswagen and the fraudulent practices of some 
investment bankers, are unfortunately numerous. In a world confronting 
serious problems that have reached a global scale, it is hard to understand 
why schools place so much emphasis on formal knowledge without 
complementary emphasis on the wisdom and ethical expertise necessary even 
to deal adequately with the problems. This issue also has been raised in 


medicine, where standardized testing has a substantial influence on admission 


to residency programs and, by extension, educational priorities for medical 
students, even though it does not capture the full scope of physician 
expertise, and could be extended to include working effectively on a team, 
communicating with patients, and managing uncertainty in a wise and ethical 
manner (Prober, Kolars, First, & Melnick, 2016). 


Developing Practical Intelligence and Tacit 
Knowledge 


Given that practical intelligence is considered a form of developing expertise 


(Sternberg, 1998a), research also has explored methods for making this 


development happen. Developmental targets have included both general 
cognitive processes and attunement to the constraints of specific 
environments. Wagner and Sternberg (1990) noted two ways to enhance 
one’s body of tacit knowledge: (1) making tacit knowledge explicit and 
sharing it, and (2) improving the ability to engage the environment and learn 
from experience. Practically intelligent behavior may also be addressed 
directly (Williams et al., 1996, 2002). 


Making Tacit Knowledge Explicit 


Although some may reject the notion of making tacit knowledge explicit, the 
potential to develop expertise by doing so has been explored and advocated 
by scholars from a variety of theoretical perspectives (Argyris, 1993; Brown 
& Duguid, 1991; Collins, 2013; Karmiloff-Smith, 1992; Schén, 1983; 
Sternberg et al., 2000; Wenger, 1998). Their central assertion is that because 


most of the relevant know-how that distinguishes different levels of expertise 
is acquired through experience, methods that stimulate the process of 
thinking about what one is doing and why, and talking about it with others, 
will facilitate the development of expertise. This developmental concept — 


acquiring expertise through transition from tacit to explicit knowing — is akin 


to that of “representational redescription” (Karmiloff-Smith, 1992), a process 
whereby implicit, domain-specific knowledge is made accessible for flexible 
application to new situations and more generalized behavioral mastery. 
Methods of making tacit knowledge explicit have sought to promote the 
verbalization of one’s experiences as condition—action narratives and to 
codify these narratives as information others can use for determining what to 
do and when (Cianciolo & Evans, 2013). 

One such method is communities of practice, defined as groups of 
people who informally come together to exchange knowledge and experience 


in a shared domain of interest (Wenger, McDermott, & Snyder, 2002). They 


are distinguished from workgroups and teams in that members are self- 
selected rather than assigned by an organizational authority, and the purpose 
of membership is to share and develop knowledge, rather than to collaborate 


on accomplishing a more specific task or assignment. Communities of 


practice have been explicitly recognized as an effective mechanism for 
developing expertise through sharing tacit knowledge (Adams & Freeman, 
2000; Cianciolo & Evans, 2013; Gherardi, Nicolini, & Odella, 1998; Lesser 
& Storck, 2001; Nonaka & Takeuchi, 1995; Wenger, 2000; Wenger & 
Snyder, 2000). 


Examples of professional or trade associations akin to communities of 


practice can be found throughout human history (Amirault & Branson, 2006; 
Lave & Wenger, 1991). Modern communities of practice are not only 
common among independent practitioners but also have formed within large 
organizations (e.g. Bobrow & Whalen, 2002; Crager & Lemons, 2003; 
Dixon, Allen, Burgess, Kilner, & Schweitzer, 2005; Lesser & Storck, 2001), 


often as part of a larger knowledge management initiative designed to 


provide competitive edge in today’s knowledge economy (Crager & Lemons, 
2003). Enabled by advances in information technology, these communities 
range from regularly scheduled meetings, listservs and online discussion 
forums (Cianciolo & Evans, 2013; Wenger, White, & Smith, 2009), to 
integrated information systems (Heredia et al., 2013). Although typically 
communities of practice are self-organizing, organizations have sponsored 
their development to cultivate needed capabilities and have enjoyed 
substantial return on investment through increased productivity and 
innovation (Crager & Lemons, 2003). 

Social learning, whereby interpersonal exchange reveals prevailing 
condition—action linkages, has been posited as the mechanism by which 
communities of practice improve tacit knowledge and expertise (Cianciolo & 


Evans, 2013; see also Panahi, Watson, & Partridge, 2012; Tee & Karney, 


2010). Contributions made to a community of practice reveal environmental 


variability and promote attunement to constraints. For example, people’s 


stories of unexpected events, the responses taken, and their outcome provide 
vicarious experience accessible to all community members that is especially 
useful in work environments predominated by non-routine tasks that limit 
exposure and outcome feedback. Detailed accounts provide the context that 
helps others understand the limits of a given story’s applicability (Denning, 
2005; Dixon et al., 2005; Tee & Karney, 2010). Summarized, these stories 
represent the community’s shared body of tacit knowledge and reveal how 
variability in the environment calls for variability in actions taken. Best 
practices and tools shared in communities of practice enable people to shape 
their work environment and act with greater practical intelligence by directly 
adopting the tacit knowledge of others. 

Knowledge elicitation techniques are another form of interpersonal 
interaction that may be used to make tacit knowledge explicit (Lintern, 
Moon, Klein, & Hoffman, Chapter 11, this volume). These techniques 
include structured and unstructured interviews, verbal protocols, stimulated 
recall, and direct observation, among others, employed to identify the 
information and strategies involved in expert performance, such as judgment, 
decision making, and problem-solving (Hoffman, Shadbolt, Burton, & Klein, 
1995; Steurer et al., 2013). Although knowledge elicitation techniques are 
agnostic regarding types of knowledge gleaned (declarative, procedural, tacit, 
and so forth), their use assumes that practical know-how is essential to 
expertise and that it may not be fully captured in policy documents and 
manuals (Lintern et al., Chapter 11, this volume). Knowledge sought by these 
techniques may be incorporated into the design of systems that support 
attunement and goal-directed behavior, of simulations that represent this 
expertise, and of instructional materials that promote further knowledge 


acquisition. 


Facilitating Tacit Knowledge Acquisition 


Facilitating tacit knowledge acquisition is a more indirect approach to 
expanding one’s body of tacit knowledge; however, it can be expected to 
have more lasting effects on one’s practical intelligence. Articulated tacit 
knowledge, shared in communities of practice or gleaned via knowledge 
elicitation, eventually becomes outdated, sometimes very rapidly, as the 
mores of cultures shift or the environment changes. Learning how to acquire 
tacit knowledge, however, never becomes outdated. 

Approaches to facilitating tacit knowledge acquisition target its three 
underlying cognitive processes: selective encoding, selective combination, 


and selective comparison (Sternberg, 1988). Specifically, instruction is 


designed to draw people’s attention to how relevant information is selected to 
guide decision making and problem-solving, how this information is 
combined to form meaningful patterns, and how prior knowledge is 
compared to new knowledge to inform decisions and action. In this way, 
cognitive conceptualizations of tacit knowledge acquisition enable a form of 


reflection on experience (Cianciolo et al., 2004; Kolb, 1984; Schon, 1983), “a 


process of guided critical thinking that directs attention selectively to various 
aspects of experience, making knowledge typically acquired without 
conscious awareness explicit and available for examination and modification” 
(Matthew & Sternberg, 2009, p. 530). 

For example, Sternberg, Wagner, and Okagaki (1993) worked with five 


groups of college students to explore how instructions differentially targeting 
the three tacit knowledge acquisition processes affected gain in tacit 


knowledge scores. Participants role-played a personnel manager evaluating 


three fictional job candidates for sales positions using three criteria: ability to 
manage themselves, to handle the tasks and problems that arise in sales 
positions, and to handle business relationships with customers, peers, and 
superiors. While completing this task, participants in one group were cued 
with relevant information and rules-of-thumb to help them make a decision 
(selective encoding). The second group received these cues but was 
additionally provided with a note-taking sheet to help link the information to 
evaluation criteria (selective combination). The third group also received the 
cues, but instead of a note-taking sheet, an evaluation completed by a 
notional predecessor in the company was provided (selective comparison). 
One control group did not complete this hiring task, and another completed 
the hiring task without instruction. Sternberg et al. (1993) found that 
participants in the two control groups showed less gain in tacit knowledge 
than the experimental groups (average gain for control and experimental 
groups was 5.6 and 15.27, respectively). Further, groups who received 
instruction addressing selective encoding and selective combination showed 
notably greater gain than the group who received instruction addressing 
selective comparison (16.8, 19.7, and 9.3, respectively). 

Matthew and Sternberg (2009) employed three training interventions 
that targeted different aspects of tacit knowledge via case-based problem- 
solving exercises. The Condition-Focused method targeted problem 
identification and goal formulation, prompting trainees to reflect on the 
situational conditions and underlying assumptions influencing their problem 
definition and goal-setting. The Action-Focused method prompted trainees to 
think about the link between their preferred action and anticipated action 
outcomes. A combined, Condition-and-Action method incorporated both 


condition- and action-focused reflection on trainees’ problem-solving 


approach (problem identification, goals, actions, and anticipated outcomes). 
In a sample of army officers, significant gains in tacit knowledge were found 
for the Condition-and-Action method relative to the focused interventions as 
well as a no-intervention control group. A positive, but non-significant effect 
of the Condition-and-Action method was found in a sample of college 
undergraduates. 

This general approach resembles other attempts to stimulate people’s 
use of environmental information and past experience to improve their 
performance. One approach, explored by medical educators (e.g. Ark, 
Brooks, & Eva, 2006, 2007; Kulatunga-Moruzi, Brooks, & Norman, 2011), 
provides learners with instructions for visually diagnosing clinical cases that 
differentially prompt feature analysis and pattern recognition. For instance, 
Ark et al. (2006) provided ten cases to learners naive to reading 
electrocardiogram images with instructions to diagnose each case by (1) 
sequentially judging the similarity of the present case to ones used in a 
previous training session, then inspecting the image for particular diagnostic 
features; (2) inspecting the image and listing all diagnostic features; (3) 
inspecting the image and listing all diagnostic features with knowledge that 
the cases were drawn randomly from the training materials; or (4) combining 
similarity judgment and feature inspection. They found that instructions to 
combine similarity judgment and feature inspection, particularly as a 
sequential process (similarity judgment, then inspection), produced greater 
diagnostic accuracy on follow-up testing. Similar results were found by 
Kulatunga-Moruzi et al. (2011) using dermatology cases. 

In another study, Ark et al. (2007) explored the impact of diagnosis 
instructions in training that involved contrasting the features of six cases: four 


example cases, a normal case, and a case with a confusable diagnosis. They 


found that upon short- and long-term follow-up testing, diagnostic accuracy 
was higher among groups who were trained using the contrastive approach, 
regardless of diagnosis instructions. This study supports the earlier findings 
of Schwartz and Bransford (1998), who took an ecological approach to 
characterizing classroom learning and argued that actively drawing learners’ 
attention to the distinctive features of cases promotes abstract thinking and 
anticipatory reasoning. 

This family of approaches is not altogether isolated from efforts to 
stimulate people’s general intellectual curiosity and critical thinking (see also 
Zimmerman, 2006). Its theoretical heritage can be traced to philosopher John 
Dewey (1933) who believed that reflective thought — the critical analysis of 
one’s ideas and behaviors — was an important aspect of good thinking skills 
and effective problem-solving. Dewey’s work has influenced the thinking of 
numerous philosophers (e.g. Ennis, 1987; Lipman, 1993) and psychologists 
(e.g. Andrade & Perkins, 1998; Bransford, Sherwood, Vye, & Rieser, 1986; 
Grotzer & Perkins, 2000; Nickerson, 1989) who have sought to understand 


and enhance problem-solving and experience-based learning. These scholars 
have used a wide range of methods to facilitate inquisitiveness and 
experience-based learning, from using stories to engage children in the 
philosophical analysis of problems (Lipman, 1993) and training the particular 
cognitive processes involved in insight and learning from context (Davidson 


& Sternberg, 1984; Sternberg, 19876), to exploring complex problems by 


discussing multiple points of view (Paul, 1987) and theories of knowledge 


(Barrows & Tamblyn, 1980). More recent research (Nakano, Muniz, & 


Batista, 2013; Tee & Karney, 2010) also has explored how environments can 
be conducive to tacit knowledge acquisition. These environmental features 


include shared concern for efficiency, trusting, collegial relationships, and 


communication facilitated by a common language and knowledge base that 


collectively drive a social experiential learning cycle. 


Developing Practical Intelligence 


In general education, differences in practical intelligence are thought to 
appear during the middle school years, setting the stage for future disparities 


in academic achievement (Williams et al., 2002). In an effort to boost school 


performance by targeting practical intelligence directly, Sternberg and his 
colleagues (Williams et al., 1996, 2002) developed the Practical Intelligence 
for School (PIFS) program. The PIFS program featured exercises in reading, 
writing, homework, and testing that were organized around five themes: 
Knowing Why, Knowing Self, Knowing Differences, Knowing Process, and 
Revisiting. For example, in the topic of writing, PIFS students were taught to 
discover why writing is important both in and out of school (Knowing Why), 
to recognize their personal strengths and weaknesses in writing (Knowing 
Self), to distinguish between different styles and strategies of writing 
(Knowing Differences), to understand the role of planning and organization 
in the writing process (Knowing Process), and to recognize the importance of 
revising (Revisiting). 

The PIFS program was implemented in Connecticut and Massachusetts 
over a two-year period. To proceed in an ecologically valid fashion, the PIFS 
researchers encouraged participating teachers to integrate PIFS instruction 
into their teaching in a manner that best suited their instructional needs. Thus, 
PIFS lessons were either infused into ongoing classroom activity or 
administered as stand-alone instruction. At the beginning and end of each 
year, teachers and researchers evaluated students’ practical thinking skills and 
academic achievement. At the end of Year 2, students participating in the 


PIFS program showed significantly greater gain in scores on practical 


assessments of reading, writing, homework, and test-taking and on academic 
assessments of reading, writing, and testing than non-PIFS students 
(Williams et al., 2002). 


Future Directions 


It is interesting that multiple lines of inquiry pursued by scholars working 
largely independently of one another nevertheless all attempt to account for 
how people accomplish feats of everyday expertise via activities that are 
learned experientially, context-bound, and secondary to goal-directed action 
in conscious awareness. The diversity of these lines of inquiry suggests a 
general dissatisfaction with an “either—or” perspective on expertise (see also 
Feist, 2013). The theory of practical intelligence and tacit knowledge 
(Sternberg, 1988, 1997; Wagner & Sternberg, 1985) emerged out of a desire 
to broaden the concept of academic and occupational expertise beyond 
general intelligence by incorporating cognitive perspectives on expert 
performance into a new, integrative way of thinking about why people differ 
(Sternberg, 1998a). Other ecological concepts of expertise were developed by 
scholars who pushed back against highly generic, yet simultaneously 
restrictive models of expertise that hampered efforts to explain or promote 
everyday expert performance (Gibson, 1979; Henry, 2006; Kirlik, 2006; 
Tolman & Brunswik, 1935). This convergence of diverse perspectives and its 
implications for making a positive difference in human achievement make 
further inquiry worth pursuing (see also Venkitachalam & Busch, 2012). 
Future research on practical intelligence and tacit knowledge should leverage 
its conceptual alignment with ecological theories in order to expand empirical 
investigation beyond the conventional methods of differential psychology 


(i.e. psychometric testing and correlational analysis). 


It would be interesting, for instance, to combine ecological and 
ethnographic methods to explore shifts between tacit and explicit knowing as 
people interact with the environment and to examine the impact of these 
shifts on performance (see e.g. Nakano et al., 2013; Venkitachalam & Busch, 
2012). This could potentially demonstrate the functional role of tacit 
knowledge by revealing when during task performance particular 
condition—action complexes are available to articulation and when they are 
subsumed by goal-directed behavior. In theory, tacit knowledge facilitates 
performance by remaining beneath awareness during goal pursuit. If tacit 
knowledge rises to awareness while performing a task, goal pursuit may be 
disrupted. This could be tested empirically and would reveal situational 
characteristics that demand tacit knowledge and foster its dissemination. For 
example, for particular communication tasks, such as sharing patient 
information during healthcare provider shift changes or planning an urban 
military raid, social judgment analysis techniques (Brehmer & Joyce, 1988) 
or repertory grids (Ryan & O’Connor, 2009) could be used to identify tacit 
knowledge a priori. Then, analyses of verbal interaction during 
communication tasks could reveal the conditions, including degree of overlap 
in participants’ tacit knowledge, under which participants must mention their 
tacit knowledge explicitly to accomplish their goals. One might expect 
increased frequency of tacit knowledge verbalization to be associated with 
decreased task performance. 

Another interesting topic to explore is how and when tacit knowledge is 


articulated during in situ instruction (see e.g. Collins, 2013; Tee & Karney, 


2010). In this context, articulation of tacit knowledge might be expected to 
improve trainees’ longer-term performance even as immediate task execution 


is made less efficient. Conversation analyses of medical teaching (Rizan, 


Elsey, Lemon, Grant, & Monrouxe, 2014; Zemel & Koschmann, 2014; 
Zemel, Koschmann, & LeBaron, 2011) provide one useful starting point for 
identifying when the articulation of tacit knowledge might be needed to 
progress task performance and the approaches that instructors could take to 
articulating the condition—action complexes that underlie successful 
performance. Here too, social judgment analysis or repertory grids could be 
used to identify tacit knowledge a priori and pinpoint its articulation in 
ongoing conversation. Information from this kind of study also could be used 
to improve clinical teachers’ ability to stimulate trainees’ tacit knowledge 


acquisition using deliberate practice techniques (e.g. Ericsson, 2015; Pusic et 


al., 2011) and providing feedback that optimizes self-regulated learning via 


environmental cuing (Butler & Winne, 1995). 


Conclusions 


Although it is delightful to be awed by artistic, athletic, or intellectual 
excellence, or struck by the brilliance of everyday adaptation, the general 
mechanisms underlying the nature and development of expertise need not be 
mysterious. The theory of practical intelligence and tacit knowledge 
(Sternberg, 1988, 1997) has provided one means for understanding expert 
performance by illuminating the complex, ever-shifting person—environment 
interaction that allows for success in everyday life. It complements the 
practical and social constructs of expertise presented in this handbook (e.g. 
Billett et al., Chapter 8; Collins & Evans, Chapter 2), while integrating 
cognitive concepts (Feltovich et al., Chapter 6). The theory’s conceptual ties 
to ecological theories of human development and behavior open up new 
doors to future research. Such inquiry would represent exciting progress in 
the quest to understand a phenomenon that has, in one form or another, 


touched us all. 
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Introduction 


The expertise reversal effect in cognitive load theory is a phenomenon that 
shows a relative deterioration in expert learning performance under some 
instructional conditions that are beneficial for less experienced, novice 
learners (Kalyuga, Ayres, Chandler, & Sweller, 2003). A significant number 
of experiments have demonstrated that more experienced participants learned 
less than expected from instruction that was very effective for novices. 
Formally, the expertise reversal effect is a statistically significant interaction 
between levels of learner prior knowledge (levels of expertise) and alternative 
instructional procedures on learning effectiveness (see Figure 40.1). That 
interaction can be either ordinal or disordinal depending on differences in 
levels of expertise. Usually, the effect does not refer to novices 
outperforming experts and it is expected that the performance of experts will 


exceed the performance of novices. 


Instructional technique A 


Instructional technique B 


Test 


scores 


Novice to expert continuum 


Figure 40.1 The interaction between levels of expertise and instructional 


techniques leading to the expertise reversal effect. 


As can be seen in Figure 40.1, with large differences in expertise, the 
interaction is likely to be disordinal. For novices, instructional procedure B 
will be better than A, but for experts, A will be better than B. In other words, 
the graphical lines will cross at some point as depicted in Figure 40.1. With 
reduced differences in levels of expertise, the cross-over point may not be 
reached and so only one or other side of the disordinal graph will appear and 
the interaction will be ordinal. With reference to Figure 40.1, if novices are to 
the right of the cross-over point, both novices and experts will obtain higher 
test scores using instructional technique A but there may be a significant, 
ordinal interaction. Alternatively, if experts are to the left of the cross-over 
point, both novices and experts will obtain higher test scores using 
instructional technique B. In both cases it is assumed that linear trend lines 
would result in a full reversal provided there was sufficient differentiation 
between novices and experts. 

This interaction usually is interpreted from a cognitive load perspective. 
For example, detailed spoken explanations presented concurrently with a 
technical diagram could be more effective for novice learners than exploring 
the diagram only. However, for relatively more experienced learners in the 
domain, studying relatively more complex diagrams in the same task area 
could be more effective when the diagrams are not provided with 
explanations, thus reversing the learning effect of diagrams with explanations 
(Kalyuga, Chandler, & Sweller, 2000). From a cognitive load perspective, the 
effect is usually explained by the unnecessary additional working memory 
load imposed on the more experienced learners when they are forced to study 


instructional explanations that are redundant for them. Measures of cognitive 


load (e.g. subjective ratings of learning difficulty or mental effort) frequently 
are used to support a cognitive load interpretation of the effect. 


Since the original overview of the effect (Kalyuga et al., 2003), the 


research in this area has expanded rapidly. Many instructional techniques 
have been investigated using various participants from primary school to 
university students in very diverse subject areas (e.g. Kalyuga, 2007, 2014; 


Kalyuga, Rikers, & Paas, 2012; Sweller, Ayres, & Kalyuga, 2011, for recent 


overviews). Similar effects have been found in studies of medical expertise 
(the intermediate effect in clinical case studies, e.g. Rikers, Schmidt, & 
Boshuizen, 2000; Schmidt & Boshuizen, 1993), and in research on the 


execution of movement when playing sport (explicit monitoring effect, e.g. 


Beilock & Carr, 2001). For each of these effects, successful expert 
performance was disrupted by inappropriate training procedures, while the 
performance of novices was enhanced. The following sections of this chapter 
present a brief review of the expertise reversal effect in cognitive load theory 
and discuss the suggested underlying mechanism of the phenomenon. In 
addition, we consider possible extensions of the effect to areas of 
professional expertise such as sport and medical practice, as well as 
implications of these findings for improving the development of expertise. 

It should be noted that the area of research discussed in this chapter 
varies somewhat from other research on expertise and expert performance 
considered in this handbook. Most of the work in this chapter concerns the 
acquisition of a limited range of skills and expertise. More advanced students 
are classed as experts and are compared to low prior knowledge students. 
Expertise is assumed to constitute a continuum rather than a novice—expert 
dichotomy. In some cases, where professional expertise that has been 


obtained over many years is considered in this chapter (e.g. expert medical 


practitioners in the intermediate effect or expert sports people in the explicit 


monitoring effect), it is clearly noted. 


Cognitive Load Theory as a Framework for 
the Expertise Reversal Effect 


Cognitive load theory is an instructional theory that investigates instructional 
design implications of major characteristics of human cognitive architecture. 
Those instructional design implications have led to a considerable range of 
instructional procedures, including the expertise reversal effect. Without the 
cognitive architecture used by cognitive load theory, we would be left with a 
series of unrelated, isolated instructional effects. Our knowledge of human 
cognitive architecture was used to generate the hypotheses that led to 
cognitive load effects, including the expertise reversal effect. Data need 
theories as much as theories need data. 

In recent years, cognitive load theory has used evolutionary psychology 
to provide an appropriate framework for analyzing those aspects of human 
cognitive architecture relevant to instruction. Evolutionary theory has been 
used in two distinct ways. First, evolutionary biology has been used to 
categorize the information processed by humans, and second, the information 
processing characteristics of evolutionary theory have been related 
analogically to human information processing. We will begin with categories 


of knowledge. 


Geary’s Evolutionary Educational Psychology 


Geary (2005, 2007, 2008, 2012; Winegard, Winegard, & Geary, Chapter 4, 
this volume) has made a distinction between biologically primary and 
secondary knowledge. Primary knowledge applies to information we have 
specifically evolved to acquire. There are several important characteristics of 


biologically primary knowledge. 
(a) It is modular with little or no relation between the modules. 


(b) Despite the immense complexity of many of the skills, primary 
knowledge is acquired easily, automatically, and largely unconsciously 
merely by membership of a functioning society. It does not need to be 
explicitly taught and indeed, because we acquire the knowledge and 


skills independently of tuition, it usually cannot be taught. 


(c) Despite its modular nature, primary knowledge includes generic 
cognitive skills (Tricot & Sweller, 2014). For example, we do not have 
to teach students general problem-solving skills that apply to any 
problem because again, people acquire those skills automatically 


without tuition. 


From an evolutionary perspective, biologically primary skills are far too 
important not to be acquired and so we have specifically evolved to acquire 
them. While we are likely to practice the skills throughout our lives, we do so 
without conscious effort or deliberate practice (Ericsson, Krampe, & Tesch- 
Ro6mer, 1993). 

In contrast to biologically primary knowledge and skills that are 


universal to all normally functioning humans, secondary knowledge is 


culturally specific and so varies between cultures. Acquiring the substantive, 
biologically secondary knowledge usually requires conscious effort, 
deliberate practice, and explicit tuition. 

Almost everything that is taught in education and training institutions 
constitutes biologically secondary knowledge and skills. Such institutions 
were invented precisely because we do not acquire these skills automatically. 
They need to be explicitly taught and consciously practiced. The cognitive 
processes associated with acquiring biologically secondary information are 


central to cognitive load theory and are discussed next. 


Natural Information Processing Systems and Human Cognitive 
Architecture 
The cognitive architecture used to process biologically secondary information 
is an example of a natural information processing system (Sweller & Sweller, 
2006). Evolution by natural selection provides an analogous example. Natural 
information processing systems can be described in terms of five basic 


principles. 


Information Store Principle 


In order to function in a highly variable environment, natural information 
processing systems require large stores of information. In the case of 
biological evolution, a genome provides that store. While there is no agreed 
method for measuring the size of a genome, all potential measures suggest 
that even small genomes, such as that for the typhus bacterium with about 
one million base pairs, are very large. The human genome has about three 
billion base pairs. 

Human long-term memory provides a functional equivalent of a 
genome. Beginning with research by de Groot (1965) and Chase and Simon 
(1973) on chess expertise, cognitive science has provided overwhelming 
evidence for the critical role of our knowledge base in cognition and learning. 
As is the case for genomes, the size of long-term memory also has no 
adequate measure but it similarly seems to be very large. Simon and 
Gilmartin (1973) estimated that chess grand masters have learned to 


recognize tens of thousands of board configurations. All substantive, 


biologically secondary areas are likely to require similarly large amounts of 


information stored in long-term memory (Ericsson & Charness, 1994). 


Borrowing and Reorganizing Principle 


The large amounts of information found in information stores are obtained by 
borrowing from other stores. Asexual and sexual reproduction provide the 
mechanisms in the case of genomes with most information borrowed from 
ancestors and reorganized during sexual reproduction. In the case of long- 
term memory, we obtain most of our stored, biologically secondary 
information from other people. We imitate what other people do (Bandura, 
1986), listen to what they say, and read what they write. It should be noted 
that while the information being transferred to long-term memory can be 
biologically secondary, the ability to obtain information from others via 
imitation, listening, or even reading is biologically primary. We have evolved 


to obtain information from others and do not need to be taught to do so. 


Randomness as Genesis Principle 


While most of the information held in an information store is borrowed from 
other stores, information needs to be created in the first instance. In the case 
of a genome, random mutation provides the creative engine. All variation 
between individuals whether or not they belong to the same species can be 
sourced to a series of random mutations. Random mutation is a random 
generation and test process. Adaptive mutations are retained while non- 
adaptive mutations are jettisoned. 

Random generate and test is similarly central to human creativity during 


problem-solving (Sweller, 2009). When faced with a novel problem requiring 


moves that cannot be generated from information in long-term memory, the 
alternative is to randomly generate moves and test them for effectiveness. 
Effective moves are retained while ineffective ones are jettisoned using 
random generate and test. The process of random generate and test is 
biologically primary and so does not need to be taught although learners may 
need to be informed that random generate and test is an important process in 
solving particular types of biologically secondary problems (Youssef-Shalala, 
Ayres, Schubert, & Sweller, 2014). 


Narrow Limits of Change Principle 


The randomness as genesis principle has structural consequences. There are 
3! = 6 permutations of 3 elements but 10! = 3,628,800 permutations of 10 
elements. Random generation becomes progressively more difficult with 
even small increases in the number of elements that are being processed. The 
number of new elements that can be processed is very small ensuring that 
large changes to an information store do not occur. In the case of a genome, 
the epigenetic system controls the number and location of mutations and 
ensures that changes to a genome are limited. 

Working memory and its processing limitations play the same role in 
human cognition and are central to cognitive load theory (see Sweller et al., 


2011; Van Merriénboer & Sweller, 2005). Working memory, as a conscious 


processor of information within the focus of attention, is severely limited in 
capacity and duration when dealing with novel information (Baddeley, 1986; 


Miller, 1956). It is usually overloaded if more than a few units are processed 


simultaneously. Even though these processing restrictions are commonly 


viewed as a negative factor in learning, they may play a fundamental role in 


human cognitive architecture by effectively preventing rapid, significant 
changes to the knowledge base, thus maintaining the functionality and 
relative stability required to guide our cognitive behavior in complex 


environments (Sweller et al., 2011). 


Environmental Organizing and Linking Principle 


This principle justifies the preceding principles. Once information has been 
stored, it can be used to generate action that is appropriate to the 
environment. In the case of biological evolution, the epigenetic system 
provides the necessary machinery. Environmental signals are used to switch 
genes on or off. For example, the skin and liver cells of any given human 
have an identical genetic component in their nuclei. The vast structural and 
functional differences between the two types of cells are due to the epigenetic 
system which uses environmental cues to switch genes on or off. 

Working memory analogously uses the environment to determine which 
knowledge held in long-term memory is used to determine action. Working 
memory, using environmental signals, can access knowledge structures in 
long-term memory to allow us to effectively function despite the severe 
processing limitations of working memory by encapsulating many elements 
of information into larger knowledge-based units that are then treated as 
single elements in working memory (Ericsson & Kintsch, 1995). This 
extensive, well-organized knowledge is the defining characteristic of 
expertise. The sheer role of knowledge in cognition which allows 
comfortable functioning in complex environments without constantly 
experiencing significant levels of mental effort drives us to become experts in 


specific task domains. 


In summary, as a consequence of fundamental characteristics of human 
cognitive architecture, the level of learner prior knowledge or expertise 
represents the most important factor influencing activity, including the 
effectiveness of different instructional procedures and techniques, which 


represents the essence of the expertise reversal effect. 


Empirical Evidence for the Expertise Reversal 
Effect 


This section reviews empirical studies generated by cognitive load theory 
demonstrating that effective instructional methods for novice learners can be 
detrimental for relatively more experienced learners and vice versa. Despite a 
variety of instructional methods and formats investigated in these studies, the 
varying level of explicit instructional support provided to learners with 
different levels of prior knowledge was the main line of research. Therefore, 
the expertise reversal effect has also been related to research on optimizing 
instructional assistance (e.g. the “assistance dilemma” of Koedinger & 


Aleven, 2007) aimed at establishing an appropriate balance between 


providing or withdrawing assistance. The vast majority of the studies 
described below used complex (high element interactivity), biologically 
secondary, domain-specific information that imposed a heavy working 
memory load for novices that could be reduced by increasing levels of 
explicit guidance. For more expert learners, the same information had 
reduced levels of element interactivity due to knowledge held in long-term 


memory. 


Worked Examples vs. Unguided Problem-Solving 


Some of the initial studies of the effect (e.g. a series of longitudinal studies 


reviewed in Kalyuga et al., 2003, in which novice learners were gradually 


trained to become experts in a specific class of tasks) investigated cases of 
expertise reversal with instructional guidance (worked examples) provided to 
learners prior to independent problem-solving. (For recent reviews of the 
worked example effect, see Renkl, 2014a, 2014b.) For example, verbal 
explanations of technical diagrams (Kalyuga, Chandler, & Sweller, 1998, 
2000) were compared to diagrams without textual explanations; or worked 
examples of how to use a database program or to program industrial 
equipment were compared to minimally guided problem-solving or 


exploration of the task (Kalyuga, Chandler, & Sweller, 2001a; Kalyuga, 


Chandler, Tuovinen, & Sweller, 2001b). These studies demonstrated that for 


low knowledge learners, studying worked examples resulted in better post- 
test results than minimally guided forms of instruction, but the difference 
disappeared or reversed for higher knowledge learners or when the same 
learners acquired more knowledge in the domain. Judging by the participants’ 
reported subjective ratings of cognitive load, unguided learning placed higher 
cognitive demands on novice learners than explicitly guided instruction. 
However, for more knowledgeable learners, exploring and solving tasks that 
were relatively new for them could be less demanding than studying 
information that included detailed step-by-step instructional guidance that 
was redundant for learners at this level. 

As longitudinal studies tend to be difficult to organize and control, most 


evidence for the expertise reversal effect has been collected in cross-sectional 


studies that compared separate groups of novices and experts. In such studies, 
the patterns of results similar to those described above were found by 
Kalyuga and Sweller (2004) in high-school coordinate geometry and 
Reisslein, Atkinson, Seeling, and Reisslein (2006) with university 
engineering students. Leppink, Broers, Imbos, van der Vleuten, and Berger 
(2012) demonstrated that low prior knowledge students achieved better 
conceptual understanding by studying worked examples of statistical 
arguments, while higher prior knowledge students learned more by 
formulating arguments themselves and answering open-ended questions. Rey 
and Fischer (2012) demonstrated that while instructional explanations 
accompanying illustrated introductory text and a computer program about 
Statistical data analyses enhanced transfer performance of novices, they 
actually inhibited performance of higher prior knowledge learners. 

While most of the initial studies on expertise reversal were conducted in 
well-defined technical domains, recently, similar patterns of results have been 
obtained in some ill-defined task areas. For example, in a series of 
experiments involving university students with different levels of prior 
domain knowledge, Kyun, Kalyuga, and Sweller (2013) investigated the 
relative effectiveness of worked examples and unguided practice in writing 
essays in English literature. The worked example condition included model 
essays that provided several possible answers to the essay question (followed 
by similar essay questions to answer), while the problem-solving condition 
included only the equivalent number of essay questions to answer with no 
model answers provided. For participants with advanced levels of prior 
knowledge, there was no benefit of worked examples. For less 
knowledgeable students (at an intermediate level), worked examples resulted 


in superior performance on the retention post-test only. Finally, for novice 


students, worked examples were also superior on a near transfer post-test but 


with no difference found on a far transfer test. 


Process vs. Product Oriented Examples 


Van Gog, Paas, and Van Merriénboer (2008) compared the effectiveness of 
traditional, product-oriented worked examples that include only step-by-step 
solutions without explaining why each step is used and process-oriented 
worked examples that provided such explanations. The results indicated that 
process-oriented worked examples could be effective for novice learners 
during the initial stages of training, but as the learners became more 
knowledgeable, the process-related explanations became redundant causing 
an expertise reversal effect. In a related study, Bokosmaty, Kalyuga, and 
Sweller (2015) showed that the addition of process-related guidance in 
geometry worked examples may inhibit performance of more experienced 
learners. The study compared an experimental condition in which students 
were provided with the solution steps and the theorems used to justify the 
steps with a condition in which only the steps to reach the answer were 
provided, with no theorems explaining the steps (problem-solving without 
any guidance was used as a control condition). The results indicated that 
relatively more experienced students who had already learnt the relevant 
theorems benefited most from step-only guidance, while for novice students 
who had not yet fully learnt the theorems, providing both the theorems and 


step guidance was effective. 


Embedded Explanations/Prompts vs. Text Only 


There have been many studies on the expertise reversal effect that involved 
partial forms of guidance. For example, when learning from textual materials, 
embedded glossaries or explanatory notes in the text (e.g. placing them above 
or below the relevant parts of the text) were more effective than the 
traditional way of placing them at the end of the whole text for novice 
learners but not for more experienced learners (Oksa, Kalyuga, & Chandler, 
2010; Yeung, Jin, & Sweller, 1998). Niickles, Hiibner, Diimer, and Renkl 


(2010) investigated the effectiveness of supporting prompts in using 


cognitive and metacognitive strategies for writing learning journals as a 
means for follow-up reflecting on the previously studied material. Initially, 
the prompt group showed higher learning results and applied more 
metacognitive strategies than the no-prompt group. However, when learners 
acquired more knowledge in the domain, the number of applied strategies 
decreased in the prompt group and increased in no-prompt group. Eventually, 
the no-prompt group outperformed the prompt group, as the prompts 
probably became redundant and interfered with learning. Similar results were 
obtained by Roelle and Berthold (2013) who demonstrated that prompts to 
focus on processing instructional explanations in management theory 
improved learning of conceptual knowledge for novice learners but impeded 
learning for more experienced learners. Subjective ratings demonstrated 


higher levels of cognitive load for experts presented with prompts. 


Added Structure vs. Unstructured Formats 


The degree of structure in interactive learning environments is a form of 
instructional guidance. Amadieu, Tricot, and Mariné (2009a) and Amadieu, 
van Gog, Paas, Tricot, and Mariné (2009b) demonstrated that low prior 
knowledge participants learned more from hierarchically structured, non- 
linear hypertext-based concept maps in biology, while high prior knowledge 
learners benefited more from unstructured formats that showed only the 
network of relations. For example, low prior knowledge learners gained more 
conceptual knowledge and indicated less cognitive load while studying from 
structured maps than from unstructured maps. A similar finding in relation to 
the degree of structure was reported by Spanjers, Wouters, van Gog, and Van 
Merriénboer (2011) who showed that novices profited from segmentations of 
animations, while more expert learners profited more from continuous 


animations. 


Added Visuals vs. Symbolic Only Presentations 


Lee, Plass, and Homer (2006) and Homer and Plass (2010) demonstrated 
expertise reversal effects by comparing two modes of representation for 
learner-manipulated variables in interactive simulations of gas laws for 
secondary-school chemistry students: a traditional symbolic representation 


bb) ce 


using words (e.g. “temperature,” “pressure,” and “volume”) along with 
corresponding numerical values and a modified format with additional visual 
representations (e.g. flames to represent temperature, weights to represent 
pressure, etc.) that were used as a form of instructional support. Results 
indicated that only low prior knowledge students benefited from the added 
visuals. High prior knowledge learners performed better following instruction 


with abstract, symbolic only representations. 


Feedback vs. No Feedback 


Another form of instructional support is feedback provided to learners during 
their problem-solving. Krause, Stark, and Mandl (2009) demonstrated that 
low prior knowledge college students learning statistics benefited more from 
worked examples when they received explicit feedback during the following 
problem-solving practice. In contrast, no effect of feedback was demonstrated 
for higher prior knowledge students. Fyfe, Rittle-Johnson, and DeCaro 
(2012) showed that primary school students with low levels of prior 
knowledge benefited from feedback during mathematics problem-solving 
followed by explicit conceptual instruction, while students with higher levels 
of prior knowledge learned better without such feedback. 

Using a computer networking training simulation with university 
students, Nihalani, Mayrath, and Robinson (2011) demonstrated that novice 
learners benefited more from learning individually with feedback than from 
learning collaboratively with similar feedback, while higher prior knowledge 
students benefited more from collaborative learning with feedback than from 
individual feedback. From a cognitive load viewpoint, feedback or hints 
provided to learners in response to real-time evaluation of their problem- 
solving performance in interactive simulations or intelligent tutoring systems 
represent forms of partial instructional support. Brunstein, Betts, and 
Anderson (2009) demonstrated that novice learners benefited from such 
explicit partial support during problem-solving in an intelligent algebra tutor, 
but for relatively more experienced learners, problem-solving with minimal 


guidance was more effective. 


Isolated Components vs. Interacting Components 


Blayney, Kalyuga, and Sweller (2010) and Pollock, Chandler, and Sweller 
(2002) demonstrated that presenting complex information as a restricted set 
of isolated components during the initial phase of instruction prior to 
providing all the fully interactive components could be an effective 
instructional method for novice learners due to a reduction in their initial 
levels of cognitive load (isolated—interactive elements technique). However, 
this technique may not have any benefits and may even be harmful for 
learners with higher levels of prior knowledge who do not require such 
artificial simplifications but would need to reconcile them with their available 
knowledge, thus consuming additional working memory resources. In a 
related study, Clarke, Ayres, and Sweller (2005) found that students with 
poor knowledge of spreadsheets learned mathematics better when instruction 
on how to use spreadsheets was provided prior to applying this knowledge in 
learning mathematics. Presenting such instructions concurrently may 
overload learners. However, students with more experience in using 
spreadsheets learned better when the required new spreadsheet skills were 


learned simultaneously with mathematical concepts. 


Summary 


The above examples of expertise reversal studies (see Table 40.1 for an 
overview) demonstrate that the effect has been obtained in many different 
domains with a range of participants from primary school to university. 
Importantly, as a cognitive load effect, it applies only to relatively complex 
tasks and materials that impose high levels of cognitive load on potential 
learners. Historically, the expertise reversal effect can be related to earlier 
(mostly 1960s—1970s) studies of aptitude-treatment interactions (e.g. 
Cronbach, 1967; Cronbach & Snow, 1977). Those studies demonstrated 


differential learning results of various treatments depending on such learner 


characteristics (aptitudes) as knowledge, skills, learning styles, personality 
characteristics, etc. Although some of the studies specifically investigated 
learner prior knowledge as an essential learner aptitude using this approach 
(Tobias, 1976, 1989), most of them focused rather narrowly on selected 
instructional techniques without considering underlying cognitive processes 
and possible explanatory mechanisms (such as working memory processes in 


cognitive load theory). 


Table 40.1 Different categories of expertise reversal reported within the 


context of cognitive load theory 


Type of expertise Sources 
reversal Brief description (examples) 


Worked examples vs. Worked examples benefited Kalyuga et al., 
unguided problem- low knowledge learners 1998, 2000, 2001a, 
solving more than unguided forms of 2001b; Kalyuga & 


Embedded 
explanations/prompts 
vs. text only 


Process-oriented vs. 
product-oriented 
worked examples 


Added structures vs. 
unstructured formats 


Added visuals vs. 
symbols only 


instruction, but the 
difference disappeared or 
reversed for higher 
knowledge learners 


Embedded explanatory notes 
or supporting prompts are 
effective for novice learners 
but not for more experienced 
learners 


Process-oriented worked 
examples (explaining why 
each step is used) are 
effective for novice learners, 
but product-oriented 
examples (providing only 
step-by-step solutions) are 
effective for more 
knowledgeable learners 


Low prior knowledge 
learners benefit from 
structured formats (e.g. 
concept maps, animations), 
while high prior knowledge 
learners benefit more from 
unstructured formats 


Low prior knowledge 
learners benefit from the 
added visuals, while high 


Sweller, 2004; 
Reisslein et al., 
2006; Leppink et 
al., 2012; Rey & 
Fischer, 2012; 
Kyun et al., 2013 


Yeung et al., 1998; 
Niickles et al., 
2010; Oksa et al., 
2010; Roelle & 
Berthold, 2013 


Van Gog et al., 
2008; Bokosmaty 
et al., 2015 


Amadieu et al., 
2009a, 2009b; 

Spanjers et al., 
2011 


Lee et al., 2006; 
Homer & Plass, 
2010 


prior knowledge learners 
benefit from abstract, 
symbolic only 


representations 
Feedback vs. no Explicit feedback during Brunstein et al., 
feedback problem-solving practice 2009; Krause et 
benefits low but not high al., 2009; Fyfe et 
prior knowledge learners al., 2012 


Novice learners benefit more Nihalani et al., 
from learning individually 2011 

with feedback, while higher 

prior knowledge students 

benefit more from 

collaborative learning with 


feedback 
Isolated components Presenting complex Pollock et al., 
vs. interacting information as a set of 2002; Clarke et al., 
components isolated components during 2005; Blayney et 
the initial phase of al., 2010 


instruction is effective for 
novice learners, but not for 
learners with higher levels of 
prior knowledge 


While most expertise reversal effect studies based on cognitive load 
theory have been conducted in technical and academic domains, there have 
been similar phenomena observed in other areas: the intermediate effect in 


the encapsulation theory of medical expertise and the explicit monitoring 


theory of sensorimotor skill. The relevant research studies in these two 


domains are briefly reviewed in the following two sections. 


Expertise Reversal in the Medical Field 


Studies of medical expertise have demonstrated that less experienced medical 
students may, under some conditions, outperform relatively more 
experienced medical practitioners on recall of specific cases. Also, research 
studies in the execution of movements in sport have shown that while novices 
performed well under well-monitored, skill-focused, and accuracy-aimed 
training conditions, such conditions could inhibit the performance of more 
expert athletes. These effects share some common characteristics with the 
expertise reversal effect. They all concern expert—novice differences in 
performance that imply that there are activities or tasks that are more 
appropriate for persons with specific levels of expertise. For each of these 
phenomena, there has been a suggested mechanism that inhibits experts’ 
performance while enhancing novices’ performance. Kalyuga et al. (2012) 
suggested that it was possible to approach the intermediate effect and the 
explicit monitoring effect as manifestations of the expertise reversal effect in 
cognitive load theory. This section briefly describes these effects by focusing 
on their common characteristics and similar underlying mechanisms, and 
relates these phenomena to the expertise reversal effect in cognitive load 
theory. 

Research on the development of medical expertise has demonstrated that 
medical experts do not always outperform less experienced individuals, at 
least on some measures. According to the intermediate effect in clinical case 
studies (e.g. Rikers et al., 2000; Schmidt & Boshuizen, 1993), advanced 


medical students remembered more and gave more elaborate explanations for 


the signs and symptoms after reading a text describing a patient’s clinical 
case than either less advanced students or experienced physicians, even 
though the experienced physicians outperformed medical students on 
diagnostic performance. For example, Vink et al. (2016) have recently 
demonstrated that concept maps constructed by medical resident groups were 
more elaborated than those of more experienced groups — they used 
significantly more links between clinical and basic science concepts, and 
these links connected basic science concepts with a greater variety of clinical 
concepts. Although residents did not use more basic science concepts, they 
used them more frequently to organize the clinical concepts. 


A knowledge encapsulation theory (Schmidt & Boshuizen, 1992, 1993; 


Schmidt, Boshuizen, & Hobus, 1988) was used to explain the phenomenon. 
While students usually have to engage in detailed and effortful biomedical 
reasoning to come up with a diagnostic hypothesis, experts have this 
biomedical knowledge implicitly encapsulated into their procedures. Using 
the cognitive architecture of cognitive load theory, students need to use 
working memory to devise a solution while experts have an encapsulated, 
domain-specific solution stored in long-term memory and retrieved using the 
environmental organizing and linking principle. Experts may arrive at a 
diagnostic decision without explicitly relying on biomedical knowledge. 
However, they may also perform worse than students on recall and 
explanation tasks, because instead of detailed and elaborated explanations 
they typically use condensed explanations that encapsulate specific concepts. 
While medical students literally reproduce the information in a case with all 
detailed biomedical knowledge included, experts use clinical concepts with 
encapsulated biomedical knowledge in their condensed reports resulting in 


less accurate and detailed recall and explanations. However, biomedical 


knowledge is still used by the experts when they encounter unfamiliar 
problems for which encapsulated knowledge has not yet been developed 
(Schmidt & Boshuizen, 1993). 

Many studies have demonstrated this effect using free recall tasks (e.g. 
Boshuizen & Schmidt, 1992; Rikers et al., 2000), pathophysiological 
explanations (e.g. Schmidt & Boshuizen, 1993; Van de Wiel, Boshuizen, & 


Schmidt, 2000), and thinking aloud protocols while diagnosing a clinical case 


(e.g. Schmidt & Boshuizen, 1993). The intermediate effect seems to be a 


robust phenomenon demonstrating that encapsulated knowledge is an 
important feature of a medical expert. In all these studies, recalled 
information, explanations, and _ think-aloud protocols included more 
encapsulated concepts, higher-level representations, and less biomedical 
concepts with increased expertise. 

For example, Schmidt and Boshuizen’s (1993) study showed that 
experts had the highest performance on the task that captured expertise, 
namely the accuracy of diagnosis. The doctors relied on a higher-level 
representation, where detailed information that was not consequential for 
diagnosis was not encoded, similar to the proposal by Ericsson and Kintsch 
(1995), Patel and Arocha (1993), and Patel, Arocha, and Kaufman (1994). 
Experts recalled more information with short presentation times and most 
importantly they encoded more of the clinically critical information for the 
diagnosis. The experts performed worse on recall and explanations if judged 
by the simple count of propositions, but not if based on the extraction of the 


information relevant to the diagnosis and the structure of the explanations for 


clinically relevant findings (Patel & Arocha, 1993; Patel et al., 1994). 


Expertise Reversal in Training Sensorimotor 
Skills 


The sensorimotor expertise reversal effect has been demonstrated in 
situations where explicit knowledge processed through working memory 
facilitated the execution of new skills by novices but interfered with already 
established skills of more experienced performers that do not heavily rely on 


working memory processes (e.g. Beilock, Bertenthal, McCoy, & Carr, 2004). 


Explicit task-relevant knowledge that can be controlled in a step-by-step 
manner is usually beneficial to novice learners, even though it might require 
significant cognitive resources. When this knowledge becomes automated 
and implicit through extensive deliberate practice, it operates effortlessly 
without consuming working memory resources. However, research on the 
explicit monitoring effect has demonstrated that activating explicit 
knowledge in working memory may interfere with the execution of automatic 


procedures (Beilock & Carr, 2001). The automated application of expert 


skills can be disrupted and reduced to a novice-like level when experts’ 
attention is deliberately focused on step-by-step performance. Automated 
sensorimotor skills (e.g. golf putting or soccer dribbling) have been inhibited 
by increased attentional control (Beilock, Jellison, Rydell, McConnell, & 


Carr, 2006). Similar deteriorations in performance occur under pressure 


(“choking under pressure” phenomenon). Pressure increases anxiety about 
correct steps and consequently increases the controlled attention to step-by- 


step skill execution (Beilock & Carr, 2001). 


In recent research on motor tasks, such as golf putting, think-aloud 
protocols have been used to track participants’ thought processes (e.g. 


Whitehead, Taylor, & Polman, 2016; see also Ericsson, Chapter 12, this 


volume). Asking participants to pay attention to specific details may change 
their thought processes and thus change their performance, for example 
accuracy of putts in golf putting. When the performance task forces a 
performer to change their processes and attend to other things than they 
would normally do, the change to their cognitive processes results in impeded 


performance. 


Underlying Mechanism and Theoretical 
Models 


The suggested theoretical explanations for the expertise reversal effect within 
cognitive load theory, the intermediate effect in the encapsulation theory of 
medical expertise, and the explicit monitoring theory of sensorimotor 
expertise are essentially the same. In all these theories, repeated practice in 
solving problems in the domain leads to automated knowledge structures that 
are applied unconsciously. Experts usually do not operate with detailed 
concepts and processes but rather activate suitable knowledge to generate an 
effective solution, diagnosis, or motor movement execution. When they 
consciously monitor the execution of well-learned or automated complex 
skills, their performance may be inhibited. 

According to cognitive load theory, novice learners use mostly random 
generate and test chains via the randomness as genesis principle when dealing 
with unfamiliar problems in the absence of suitable knowledge in long-term 
memory. In the encapsulation theory of medical expertise, intermediate-level 
participants without practical diagnostic experience also deal with mostly 
random unconnected symptoms attempting to relate them to biomedical 
concepts by way of reasoning. This elaborate processing of each case leads to 
their demonstrated better recall of details of the case compared to medical 
experts. Finally, in explicit monitoring theory, novices enhance their 
performance by consciously manipulating parts of their knowledge and 
explicitly controlling skill execution to achieve performance goals by either 


following instructions or randomly testing various possible options. 


As mentioned earlier in this chapter, in most expertise reversal studies, 
the expert performance, even under unfavorable conditions, did not actually 
deteriorate to the level of novice performance. However, some expertise 
reversal research, particularly in cross-sectional rather than longitudinal 
studies, has demonstrated actual deteriorations in expert performance similar 
to the deteriorations on some indicators of expert performance observed in 
the intermediate effect research. Overall, despite some differences between 
these effects, considering their common underlying cognitive mechanisms, it 
is possible to consider the intermediate effect in studies of medical expertise 
and the explicit monitoring effect in complex motor skills as manifestations 
of the expertise reversal effect. Considering the role of knowledge structures 
in all cognitive activities (including learning) in general, it would not 
normally be a surprise that instructional techniques or formats that work well 
for more knowledgeable learners may not be suitable for novices who may 
not have a sufficient knowledge base to make sense of them. What is more 
counterintuitive is how instructional procedures that are effective for novices 
may be disadvantageous or detrimental for more experienced learners since 
we normally expect them to be at least neutral or harmless. The expertise 
reversal effect contradicts such expectations. A suggested cognitive load 
explanation is that processing any redundant information may require 
additional working memory resources in comparison to instructions that do 
not include this information. These resources would then be unavailable to 
other relevant learning activities thus reducing potential learning outcomes. 

According to cognitive load theory, the purpose of instruction is to 
organize and store increasing amounts of complex, domain-specific, 
biologically secondary knowledge in long-term memory resulting in 


increased expertise. That knowledge reduces element interactivity and via the 


environmental organizing and linking principle, reduces working memory 
load. Thus, the limitations of working memory disappear for high expertise 
learners when dealing with previously learned information stored in long- 
term memory. 

Furthermore, if novice learners without a relevant knowledge base are 
not provided with appropriate instructional support, they need to use search- 
based problem-solving approaches based on the randomness as genesis 
principle rather than the borrowing and reorganizing principle that is 
specifically designed for novices to acquire novel information. These search- 
based approaches are not efficient as instructional methods as they require 
significant working memory resources and may overload limited working 
memory capacity (Sweller, 1988). On the other hand, when more 
knowledgeable (more expert) learners need to process instructional guidance 
that is redundant for them, their cognitive resources may also be 
unnecessarily consumed compared to instructions without such guidance. 

For example, more expert learners may require additional resources to 
relate their available knowledge structures with the provided supporting 
information (instead of just using the available knowledge to fluently execute 
the solution steps via the environmental organizing and linking principle). 
When instruction supports construction of knowledge that learners have 
acquired previously, the redundant learning activities can unnecessarily 
increase working memory load and thus reduce relative learning outcomes 
(Kalyuga, 2007; Kalyuga & Renkl, 2010; Sweller et al., 2011). The 


assumption that knowledge construction is needed using the borrowing and 


reorganizing principle when the required knowledge already has been 
constructed is likely to be counterproductive. Rey and Buchwald (2011) 


provided direct empirical evidence supporting this cognitive load explanation 


of the effect rather than an alternative explanation based on affective, 
motivational factors which assumes that experts may not be well motivated to 
process information they already know (e.g. Paas, Tuovinen, Van 
Merriénboer, & Darabi, 2005; Schnotz, 2010). Thus, the expertise reversal 
effect in cognitive load theory is interpreted based on fundamental features of 
human cognitive architecture, primarily on the role of knowledge structures 
in long-term memory during the learning process. 

According to a broader theoretical model suggested by cognitive load 
theory, knowledge structures in long-term memory essentially perform an 
executive role in complex cognition — they direct learner attention and govern 
performance. If there is no relevant knowledge in long-term memory, 
external instruction based on the borrowing and reorganizing principle may 
substitute for knowledge. For novice learners, explicit external instruction 
may be the only source of executive function, while for experts, most of the 
required knowledge is already in long-term memory and can act as an 
executive function via the environmental organizing and linking principle. At 
intermediate levels of expertise, both sources of executive guidance 
(knowledge and instruction) may be used. If properly aligned, they should 
complement each other: long-term memory knowledge should guide 
processing of familiar information, and instruction should guide processing 
of unfamiliar information. In real situations, there may be insufficient 
guidance provided for novices to deal with new information requiring 
learners to use the randomness as genesis principle with its attendant 
cognitive load, or instructional guidance may be provided to learners who 
already have knowledge for dealing with presented information causing 


cross-referencing processes that distract learners from fluently executing 


already learned procedures. In either case, an unnecessary cognitive load may 


be imposed causing an expertise reversal effect. 


Implications for the Acquisition of Expertise 


It is a well-known principle of the cognitive approach to educational 
psychology that learner prior knowledge is the single most important factor 
influencing learning, and it should always be considered when teaching 
(Ausubel, 1968). The general implication of the expertise reversal effect for 
the design of instruction in different domains fully confirms this principle. It 
also asserts that the effective selection of specific instructional or training 
methods depends on levels of learner expertise. However, the studies of the 
expertise reversal effect within cognitive load theory have further specified 
this general implication by suggesting concrete methods for how we can 
teach based on learner prior knowledge. This research has provided empirical 
support for prescribing particular instructional methods for novices and for 
more experienced learners. 

For example, it has been established that novice learners benefit more 
from explicit instruction that directs their attention to step-by-step 
procedures. In contrast, relatively more experienced learners benefit from 
methods that do not explicitly focus their attention on performing already 
acquired or automated steps and instead allow them to explore the task 
(Kalyuga et al., 2000). Thus, the selection and sequencing of learning 
activities need to be tailored to learner levels of expertise in specific task 
areas. For example, comprehensive explicit instructional guidance (as a 
substitute for absent knowledge in long-term memory) should be provided to 
novices. At intermediate levels of expertise, a blend of explicit guidance and 


problem-solving with reduced guidance forms should be used (e.g. providing 


the step labels without any detailed explanations). Finally, advanced learners 
with higher levels of expertise could be provided with minimally guided 
problem-solving practice. 

Thus, the studies of the expertise reversal effect in cognitive load theory 
essentially examined the effectiveness of different types of training tasks for 
more and less skilled individuals to design instruction for attaining skill in a 
new domain. The important result of these studies is that the benefits of the 
task structure differ as a function of the acquired level of expertise (or level 
of performance) — with increases in expertise the benefits of the well- 
structured tasks are reduced. These results are in correspondence with 
research on deliberate practice (see Ericsson, Chapter 38, this volume) that 
shows the importance of individualized coaching and teaching of students 
and personalized training tasks for each individual in professional domains 
such as music, sport, and ballet. The expertise reversal research shows how 
changes in cognitive processes mediate performance. There needs to be a 
match between instructional manipulations and the processes that normally 
support execution of tasks at different levels of achievement. 

Incremental decreases in levels of instructional support with increases in 
learner expertise may be realized using a completion strategy in which 
learners are asked to complete partially completed problems (Van 
Merriénboer, 1990), or faded worked examples (Renkl, 1997) in which 
learners are initially presented with worked examples that are substituted by 
completion problems, and eventually full problems with increases in 
expertise. Other methods to decrease support and guidance with increasing 
learner expertise such as a gradual shift from system control to learner control 
over the selection of new tasks, using electronic development portfolios, or a 


gradual shift from low-fidelity to high-fidelity, real task environments are 


described in the Four-Component Instructional Design (4C/ID) model (see 
Van Merriénboer & Kirschner, 2013). Progress in the acquisition of 
knowledge needs to be monitored and instructional procedures tailored 
accordingly. Adaptive learning environments using dynamic real-time 
evaluation of levels of learner expertise associated with appropriately tailored 
instructional procedures may be the best way of implementing this approach. 
Adaptation methodologies using rapid assessment techniques in conjunction 
with the expertise reversal effect have been successfully applied using 
computer-based tutors in some task areas in mathematics and science 
(Kalyuga, 2006; Kalyuga & Sweller, 2004, 2005) and in professional 


domains such as air traffic control (Salden, Paas, & Van Merriénboer, 2006). 


Pachman, Kalyuga, and Sweller (2013) used rapid assessment to 
investigate the effectiveness of alternative practice schedules for improving 
geometry performance of secondary school students with different levels of 
prior experience. An experimental condition was based on deliberate practice 
intentionally aimed at learners’ weak areas and only their weak areas. In 
contrast, students in the control group could choose any practice problems 
and their sequences. The first experiment indicated a significant interaction 
between practice schedule and learner levels of expertise — an expertise 
reversal effect. The interaction was caused by novice learners benefiting 
more from free-choice practice than deliberate practice. The subsequent 
experiments showed that relatively more knowledgeable learners improved 
more by focusing only on their weak areas during practice, while less 
knowledgeable learners benefited more from combining some of their weak 
and some of their strong areas, or by focusing on a limited number of weak 
areas. From a cognitive load perspective, attending to all their weak areas 


could result in a working memory overload. Pachman, Kalyuga, and Sweller 


(2014) obtained similar results with high school students practicing on 
geometry tasks using worked example problem pairs (free choice of practice 
pairs vs. deliberate practice groups). The deliberate practice condition was 
effective only for relatively more knowledgeable learners. 

An important implication of these studies for the design of deliberate 
practice techniques in secondary education is that practice in weak areas 
alone should be encouraged, but only after students have acquired some 
experience in the domain. Before a sufficient level of familiarity with the 
subject matter or prior experience is achieved, learners should practice mostly 
in the areas in which they have some degree of competence, e.g. practice on a 
mixture of learnt and unlearnt procedures or focus only on a few weak areas 
at a time. Regular self-checks can be introduced to more knowledgeable 
learners to raise their awareness of the development processes associated with 
expertise. This recommendation closely relates to research on how 
(electronic) development portfolios in professional education (e.g. medical 
residency training) can assist learners in choosing new learning tasks (e.g. 


Van Merriénboer & Sluijsmans, 2009). Such portfolios collect information on 


performance over time/tasks, and help learners to identify their progress and 
points of improvement, so that they are better equipped to select future 
learning tasks or parts of the tasks to focus on. 

As was mentioned earlier in this chapter, expertise is assumed to 
constitute a continuum rather than a novice—expert dichotomy. It should be 
noted that it is not yet clear at this stage whether the reversal effects as 
described in this chapter might also hold for persons with very high, 
professional levels of expertise. A corresponding alternative hypothesis might 
be that for higher-level experts, their high level of knowledge makes them 


independent of specific instructional designs, so that they are not hampered 


by instructional designs intended for novices. Until now, there have not been 
many expertise reversal studies that included very high-level experts. In fact, 


the only such study familiar to us (Oksa et al., 2010) investigated the effect of 


integrated footnotes in Shakespearean texts but still found an actual reversal 
effect. More experiments with professionals with very high levels of 
expertise need to be conducted to see if this result can be replicated. 

In conclusion, many people intuitively assume that if an instructional 
method works well with novice learners, then it also should not be harmful 
for relatively more knowledgeable learners. The above reviewed studies 
related to the expertise reversal effect have demonstrated that this assumption 
is false. More experienced learners’ performance may actually deteriorate 
when their existing knowledge base overlaps with instructional support 
designed for novices. Removing such support may allow experienced 
learners to take full advantage of their knowledge base in further 
strengthening their skills. The effect has been demonstrated in many different 
domains with a wide range of participants, using both longitudinal and cross- 
sectional studies of novice—expert differences to test the effectiveness of 
alternative instructional procedures. Studies of expert—novice differences in 
medical expertise and the execution of movements in sport have 
demonstrated similar patterns of results indicating the possibility of common 
cognitive processes that inhibit the performance of experts while enhancing 
the performance of novices. The generalized expertise reversal effect may 
have important instructional implications for improving the acquisition of 


expertise in a broad range of fields. 
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The study of expertise has, over the last 30 years, had widespread influence 
on conceptions of cognitive functioning (e.g. Ericsson, Charness, Hoffman & 


Feltovich, 2006; Ericsson & Pool, 2016). A general conclusion from this 


research is that deep immersion in a domain plays a significant role in human 
performance at all levels. The potential role of expertise is of great interest in 
the study of creative thinking, the cognitive processes that underlie the 
production of novel ideas and products. There is, however, disagreement 
concerning the role of expertise in creative thinking. One viewpoint — the 
remote-associates perspective — building on ideas promulgated more than a 


century ago, argues that creativity and expertise are negatively related. 


Formulating new ideas requires that one break away from the past, i.e. think 
outside the box. The second view, the expertise perspective on creative 
thinking, proposes that expertise plays an important positive role in creative 
advances. My chapter in the first edition of this handbook (Weisberg, 2006a) 
reviewed evidence that expertise is important in creative thinking. This 
chapter goes further, exploring the conflicting views on the role of expertise 


in creativity. 


Outline of the Chapter 


I first present brief overviews of the remote-associates versus expertise views 
on creative thinking, followed by a more detailed discussion of the two 
viewpoints. Taking a historical perspective, I begin with the remote- 
associates view, which assumes that, in situations demanding creative 
thinking, attempts to rely on expertise will fail. In response to failure, 
mechanisms come into play that make possible combinations of ideas that 
were only remotely associated. I will discuss several of those mechanisms 
and also several case studies of creative advances presented in support of the 
remote-associates view. The next section of the chapter presents evidence for 
the expertise view, examining several case studies demonstrating that 
expertise served as the basis for major creative advances. I then examine 
challenges that the expertise view presents for the remote-associates 
perspective, and the final section of the chapter explores some broader 
questions concerning expertise and creativity. The general conclusion of the 
chapter is that — the long history of the remote-associates view 
notwithstanding — creative advances are better understood as building on 


expertise than as being the result of rejecting what we know. 


Two Perspectives on Creativity and Expertise 


Remote Associates and Creativity: Unstructured Imagination in 
Creative Thinking 
The remote-associates view assumes that novel ideas are produced by linking 
previously unconnected ideas; i.e. ideas that are remotely associated. Modern 
versions of this perspective arose from the confluence of Poincaré’s (1908/ 
2005) emphasis on remote associations in creativity and the Gestalt 
psychologists’ concepts of restructuring and insight in problem-solving (e.g. 


Kohler, 1925; Wertheimer, 1982). The remote-associates view assumes that 


creative imagination is unstructured, because creative thinking does not 
utilize associative connections or mental structures available in memory. In 
addition, proponents of the remote-associates view assume that creative 
thinking requires a relinquishing of executive control over the thought 
process (e.g. Wiley & Jarosz, 2012). I have elsewhere used the term tension 
view to refer to the perspective that argues that expertise works against 
creativity. Here I use the term remote-associates view, because it more 
specifically describes the basis for the tension between expertise and 


creativity. 


Expertise and Structured Imagination in Creative Thinking 


The expertise view assumes that presentation of a problem results in retrieval 
of knowledge — i.e. expertise — from memory; creative advances evolve out 
of attempts to apply that knowledge to the new situation. Research examining 
creativity has uncovered a critical role of expertise in innovations of many 


sorts, from solving problems in the laboratory (e.g. Fleck & Weisberg, 2004, 


2013) to larger-scale creative achievements (Ericsson, 2014; Hayes, 1989; 
Kozbelt, 2005; Weisberg, 2006a, 2006b, chs. 1, 5, 2009, 2011). The expertise 


view, in contrast to the remote-associates view, assumes that the creative 


imagination is structured, since creative thinking utilizes established 
associative connections (near associates) or mental structures. The expertise 
view also could be called the near-associates or transfer of knowledge 
perspective. According to the expertise view, one should never find a leap 
from one idea to another that is only remotely linked to it. The only way that 
one can produce a connection between distant ideas is as the result of a series 
of steps, each of which builds on what came before and is based on near 


associates. 


Remote Associates in Creative Thinking 


The remote-associates perspective is central in much modern theorizing about 
creativity. Here are two examples of that view; other examples will be 
discussed as the chapter unfolds. Wiley and her colleagues have carried out 
numerous studies examining aspects of problem-solving and creativity. In 
discussing how people solve problems requiring creative thinking, Wiley and 
Jarosz (2012) proposed that a “solution seems to require either a completely 
original approach (i.e., restructuring), or a novel combination of diverse bits 
of information through remote associations in memory” (p. 259). Similarly, 
Kounios and Beeman (2015), who have carried out an influential research 
program examining brain processes underlying insight in problem-solving, 
defined creative thinking as “the ability to reinterpret something by breaking 
it down into its elements and recombining those elements in a surprising way 
to achieve some goal ... [T]he less obvious the recombination, the more 
creative it is ... When this kind of creative recombination takes place in an 
instant, it’s an insight” (pp. 9-10). 

Both sets of researchers emphasize the notion of remote associations, 
Wiley and Jarosz use that term, while Kounios and Beeman mention that 
“less obvious” recombinations are “more creative.” As noted, this perspective 
can be traced back over 100 years, to the theorizing of Poincaré (1908). Both 
definitions also mention restructuring. Wiley and Jarosz do so explicitly, and 
Kounios and Beeman talk about reinterpreting something, which is the same 
thing. Restructuring, of course, was central in the Gestalt psychologists’ 


discussion of insight in problem-solving (e.g. Kohler, 1925; Wertheimer, 


1982). The only difference between the two definitions is that Wiley and 
Jarosz discuss restructuring and remote associations as separate mechanisms, 
while Kounios and Beeman propose that reinterpreting a problem can result 
in remotely associated ideas coming together. I now turn to a more detailed 
examination of the notion that remote associations underlie creative thinking, 
and will examine several mechanisms that have been proposed to enable 


creative thinkers to access remote associations. 


Poincaré: Unconscious Thinking and Remote Associations 


Poincaré (1908) was perhaps the first proponent of the idea that creative 
thinking depended on bringing together ideas that were “wrongly believed to 
be strangers to one another” (p. 44), i.e. ideas that were not directly 
associated. In his view, connections among those unrelated ideas were made 
in the unconscious. Poincaré reported that a similar sequence of events 
occurred in several of his creative advances in mathematics. He first worked 
unsuccessfully on a problem for a significant amount of time, before in 
frustration abandoning work and ceasing to think about mathematics. While 
away from the problem, he had an illumination, for example the realization of 
a link between two concepts. Based on those incidents, Poincaré concluded 
that the unconscious had taken up the problem where conscious thinking left 
off, and sometimes went significantly beyond it. In Poincaré’s initial attempts 
to deal with a problem, various ideas were activated, and, after he stopped 
working, some remained active, interacting with each other and, more 
importantly, with previously inactive ideas. Those continuing interactions 
could produce a new combination of ideas, involving an idea that had been 
activated combining with one that had not been considered before, that was 
only remotely associated with it. 

Wallas (1926/2014) used Poincaré’s reports, as well as others, as the 
basis for his influential four-stage model of creativity. Incubation, the second 
stage — labeled with Poincaré’s term — is most relevant to the current 
discussion; it occurred when Poincaré stopped conscious work, but work 
continued unconsciously. The fruit of that unconscious work became evident 


with an illumination — again Poincaré’s term — the unexpected flash into 


consciousness of a new idea. Poincaré’s theorizing, emphasizing the 
importance of remote associations in creative thinking, as presented in 
Wallas’s model, forms one pillar of modern theorizing. However, the 
importance of remote associations in creative thinking was brought centrally 
into psychology by Mednick (1962). He provided a mechanism — flat 
associative hierarchies — to explain the capacity of some people to produce 


creative ideas, which was in accord with thinking in mid-twentieth-century 


psychology. 


Flat Associative Hierarchies and Remote Associations 


Mednick (1962) described any situation that demanded creative thinking as a 
set of stimuli, each of which had different elements (responses) associated 
with it, and he defined creativity as the combining of associative elements 


into useful combinations. Echoing Poincaré (1908), Mednick proposed that, 


“The more mutually remote the elements of the new combination, the more 
creative the process or solution” (1962, p. 221), and that “It is among these 
more remote responses that the requisite elements ... for a creative solution 
will be lurking” (p. 223). (There are also strong similarities between 
Mednick’s definition and those of Wiley & Jarosz (2012) and Kounios & 
Beeman (2015).) In explaining how creative thinkers could utilize remote 
associations, Mednick proposed that they differed from non-creative thinkers 
in the set of responses, or the associative hierarchies, that they brought to 
problem situations. Non-creative individuals possessed a steep associative 
hierarchy, with a small number of high-frequency or dominant associations, 
which would make it difficult for the individual to think of something new. 
The creative individual, in contrast, possessed a flat associative hierarchy, 
with a relatively large number of responses of essentially equal strength. 
Such a structure would enable the individual to produce what, from the 


perspective of the non-creative individual, would be a remote associate. 


Modern Perspectives on Remote Associations 


The explanatory status of Mednick’s (1962) flat versus step associative 
hierarchies is problematic, because he did not discuss how the different sorts 
of hierarchies came about. Different types of hierarchies might reflect the 
experiences of the individuals, so that, for example, a flat hierarchy was the 
result of a set of experiences of approximately equal frequencies. The 
creative person would have a larger set of responses available because of the 
near-equal frequencies. On the other hand, it might be the case that we all 
have similar sets of experiences, but the creative individual is somehow able 
to utilize a wider set of them in dealing with a situation. Theorists adopting 
the remote-associates perspective have typically made the latter assumption, 
and have proposed several mechanisms that would allow the creative 
individual to bring together remotely associated ideas. I will consider four, 
not necessarily independent, mechanisms: cognitive disinhibition; broad 
rather than focused attention; lack of executive control; and insight through 


restructuring. 


Cognitive Disinhibition and Remote Connections 


Carson (e.g. 2014), expanding on ideas proposed by Eysenck (e.g. 1995) and 


Martindale (e.g. 1999), has presented a wide-ranging analysis of creativity, 


based on the idea that the mechanism of cognitive disinhibition separates 
creative thinkers from the non-creative. Although Carson’s research builds on 
ideas of Eysenck and Martindale, I concentrate on her research because it 
provides another recent example of the remote-associates position. Carson 
assumes that the most valuable new ideas are the result of remote 
associations, and proposes that creative people are able to bring together 
weakly connected ideas because such people are less subject to cognitive 
inhibition, the mechanism that underlies the ability to focus attention on 
stimuli relevant to the task at hand. Consider a situation in which you must 
respond to one stimulus — the target — in a group of stimuli, while not 
responding to, i.e. ignoring, the others. A new target is then designated, 
which can be either one of the previously ignored stimuli or a new one. 
Responding accurately to the new target takes longer if it is one of the 
previously ignored stimuli. Thus, ignoring a stimulus in the first part of the 
experiment involved active inhibition — cognitive inhibition — of its 
representation, so that it would not trigger a response. 

In the remote-associates view, it can be a mixed blessing to keep 
attention focused on only the task-relevant ideas (Carson, 2014). In a task 
demanding creative thinking, it is necessary to attend to ideas remotely 
related to the elements in the situation. Some people, in whom cognitive 
inhibition is low, are less able to screen ideas for relevancy. Thus, cognitive 


disinhibition — a lack of cognitive inhibition — allows those remote ideas to 


come to mind. Such people would be more likely to experience a broader 
range of ideas in any situation, and, therefore, more likely to bring together 
ideas that people high in cognitive inhibition would not have available. 
According to Carson (2014), “one mental process, cognitive disinhibition, 
may impart a qualitative difference in the ability to generate creative ideas” 
(p. 198). Carson cites several studies that have examined the relationship 
between creativity and cognitive disinhibition. People who demonstrate high 
levels of cognitive disinhibition in laboratory settings report higher levels of 
real-world creative accomplishments than do people showing low levels of 
cognitive disinhibition, supporting the role of cognitive disinhibition in 


creative thinking. 


Broad Attention and Creative Thinking 


The concept of cognitive disinhibition is closely related to the notion that 
broad, rather than narrow, attentional focus facilitates creative thinking (e.g. 
Friedman, Fishbach, Forster, & Werth, 2003; Wegbreit, Suzuki, Grabowecky, 


Kounios, & Beeman, 2012). Just as one can concentrate attention narrowly or 


broadly during perception (e.g. observing the flow of traffic on a highway, 
rather than looking for a specific model of car), one can also, when thinking, 
narrowly or broadly focus one’s attention on ideas. If one focuses narrowly, 
one attends to ideas that are strongly activated by the problem, in other 
words, to near associates to the elements in the problem. One will then be 
limited to familiar, i.e. non-creative, ideas. If one can spread one’s internal 
attention broadly, one may become sensitive to weakly activated ideas 
(remote associates), which might contribute to a creative advance. 
Researchers have proposed that directing attention during thinking uses 
the same mechanisms as directing attention during perception. Therefore, 
having people carry out a perceptual task that requires broad attention might 
facilitate creative thinking, because using that strategy during perception 
might transfer to the creativity task. Friedman et al. (2003) had people first 
carry out a perceptual task involving searching a computer display to find a 3 
among a group of random digits. In the narrow-focus condition, the digits 
were clustered near the center of the screen; in the broad-focus condition, the 
digits were in the periphery. The participants then worked on two creative 
thinking tasks. One was to think of unusual uses for a brick, and the other 
was to produce a creative title for a photograph. People produced more- 


creative responses in both creativity tasks after the broad-attention task, 


indicating that creativity depends on spreading one’s internal attention 


broadly with respect to ideas (see also Wegbreit et al., 2012). 


Lack of Executive Functioning in Creativity 


Cognitive disinhibition and broad attentional focus are closely related as 
mechanisms supporting the utilization of remote associations. In both cases, 
one assumes that people differ in their capacity to focus on ideas relevant to 
the task at hand. As noted earlier, while it may sometimes be important to 
focus attention on only task-relevant ideas, generation of a creative response 
to a situation may require that one deal with ideas remotely associated to the 


task elements. Wiley and her colleagues (e.g. Jarosz, Colflesh, & Wiley, 


2012; Wiley & Jarosz, 2012) have proposed that individual differences in 
executive functioning, as measured by working memory capacity, might 
provide a general mechanism whereby some people are more able to produce 
novel responses in some situation. In this view, solving certain sorts of 
problems — analytic problems, e.g. verbal arithmetic problems — depends on 
executive functioning. However, executive control can be detrimental when 
dealing with problems that require creative thinking. In those sorts of 
problems, it is necessary to go beyond the ideas activated by the problem, to 
get to remote ideas, and that is where a lack of executive functioning might 
play a role. Wiley and her colleagues have provided several different 
demonstrations of situations in which weak executive functioning contributes 
to creative thinking, one of which involved providing participants with 
alcoholic beverages before giving them a problem to work on (Jarosz et al., 
2012). Alcohol consumption interfered with executive functioning, but it 
facilitated performance on problems requiring creative thinking, presumably 
because “drinking appears to make attention more diffuse, allowing 


intoxicated individuals to better access remote solutions” (Wiley & Jarosz, 


2012, p. 260). This research makes clear the interconnections between several 


aspects of cognition that may play a role in creative thinking, all of which are 


assumed to be related to executive control in various ways. 


Restructuring as the Basis for Creativity 


I now turn to the idea that creative ideas arise from restructuring a 
problematic situation. The same assumption is operative here, i.e. that 
creative thinking depends on the rejection of expertise, although the 


postulated mechanism may be a bit different. 


Productive Thinking and Insight 


In the Gestalt analysis of creativity, truly novel responses could only be 
brought about through productive thinking, which enabled the thinker to go 
beyond the past (Wertheimer, 1982). Modern researchers have concentrated 
on insight arising from restructuring as one expression of productive 


thinking. Research on insight (e.g. Kounios & Beeman, 2015; Ohlsson, 2011; 


Perkins, 2000; Wiley & Jarosz, 2012) has focused on differences between 
insight and analysis as modes of thinking. Analytic thinking arises from the 
transfer of expertise to a situation, while restructuring involves mechanisms 
that allow one to break away from what one knows. When facing a problem, 
an individual will first attempt to apply responses that have previously been 
successful in similar situations, through what the Gestalt psychologists called 
reproductive thinking (equivalent to analytic thinking or transfer of 
expertise). In situations that demand creative thinking, however, those 
reproductive responses will not work, leading the thinker to a state of 
impasse. In response to impasse, the situation may undergo a restructuring — 
a reorganization or reinterpretation of the problem’s components — that may 
bring with it a sudden insight into a new way of approaching the problem, or 


even a complete solution. This series of events — failure = impasse > 


restructuring => insight > solution — is called the insight sequence (Fleck & 
Weisberg, 2013). Restructuring is another mechanism that serves the purpose 
of bringing together ideas that were only weakly, i.e. remotely, linked. 
Henceforth, I will use the terms insight or restructuring to refer to the insight 
sequence. 

I will examine three modern developments in theorizing concerning 
insight. First is Ohlsson’s (2011) discussion of deep learning: special 
cognitive mechanisms that allow us to move beyond knowledge in order 
address novel situations that our experience cannot deal with. A similar idea 


was proposed by Perkins (2000), which he called breakthrough thinking for 


an unreasonable world. Finally, Kounios and Beeman (2015), who have 
already been briefly mentioned, have carried out an extensive research 
project aimed at identifying the brain mechanisms underlying insight 
(Weisberg, 2013). On a psychological level, as we have seen, they postulate a 


critical role for restructuring in the utilization of remote associations. 


Ohlsson: Deep Learning in a Changing World 


In Ohlsson’s (2011) view, the world is in constant wide-ranging flux. 
Analytic thinking, based on expertise, cannot deal with that ever-changing 
present. Deep learning is a set of mechanisms that allow an individual to 
formulate new ways of dealing with the present, by rejecting the past. There 
are several classes of situations in which deep learning must occur. Most 
important for us are situations that demand restructuring rather than analytic 
thinking. Ohlsson proposed that only through the mechanisms of deep 


learning can we survive the large-scale changes that we constantly face. 


Ohlsson (2011) discussed Arnold Wilkins’s invention of radar in 1935 
as an example of a creative advance brought about through insight. Wilkins 
was a physicist who worked at the Radio Research Station in England, a 
government laboratory. He had been asked by his superior to determine 
whether radio waves could be used to raise the temperature of eight pints of 
water from 98° Fahrenheit to 105°, at a distance of five kilometers and a 
height of one kilometer. Given the political climate in Europe in the mid- 
1930s, with war likely, the purpose of the request was to determine the 
feasibility of using radio waves as a “death ray,” to heat the blood of pilots of 
incoming enemy aircraft, thereby disabling or killing them. However, 
Wilkins determined that the task could not be carried out with the available 
technology. Wilkins’s superior, wanting his agency to assist in the war effort, 
asked him if he could think of any way in which they might contribute. 
Wilkins thought about the request and, in Ohlsson’s words (p. 53), “[iJn a 
momentous act of inspiration,” conceived of using radio waves as a way of 
specifying the locations of incoming enemy planes, thereby providing 
information that could assist in intercepting and destroying them. In a leap of 
insight bringing together two remotely associated ideas — radio waves and 
locating aircraft — Wilkins had conceived radar. 

Edison’s invention of the light bulb was, according to Ohlsson (2011, 
p. 134), another creative advance that came about as the result of insight. The 
main problem facing Edison and his laboratory staff was the design and 
composition of the “burner” in the light bulb. This is what we call the 
filament: a thin piece of material that glows when electric current is passed 
through it, producing light. Ohlsson described Edison and his team as 
working for about a year with platinum wire, configured in various ways, as a 


possible burner. However, they were not able to design a burner that would 


glow with sufficient brightness for a long-enough period of time to be useful. 
After this period of only limited progress, Edison turned to carbon 
compounds as possible burners, and relatively quickly found success. Thus, 
one can describe Edison’s situation as first reaching an impasse (Ohlsson 
describes “a year with only partial progress”; p. 134), which resulted in a 
restructuring of the situation, i.e. changing the focus from platinum to carbon 


— presumably a remotely associated idea — where success came about. 


Perkins: “Breakthrough Thinking” in an “Unreasonable” World 


In a similar vein, Perkins (2000) proposed that creative advances at the 
highest level are brought about as a result of “breakthrough” thinking, which 
brings together ideas that might not be seen as related. Breakthrough thinking 
is necessary because sometimes the world can be an “unreasonable” place. 
Any individual faced with a problem will typically attempt to approach it in a 
“reasonable” manner, i.e. based on expertise. However, there will be times 
when that approach will not work, resulting in an impasse. Breakthrough 
thinking is then called for, with its rejection of the past. The person may 
search for ideas, and, if that search is carried out correctly — Perkins provides 
guidelines — there may be a “cognitive snap” (p. 9). In a sudden insight, a 
new idea becomes the focus of attention, bringing with it a new way of 
approaching the problem. 

Perkins discussed Leonardo’s design of the “aerial screw,” a flying 
machine, as an example of breakthrough thinking. The aerial screw (see 
Figure 41.1) is based on the idea that humans could fly — i.e. rise up in the air 
— if they could very quickly turn a large-enough “screw” (the screw-shaped 


sail on the machine). Just as a wood screw can be driven into a wooden 


ceiling, the aerial screw can be driven into the air. Perkins emphasized 
Leonardo’s leap of analogical thinking that connected two remotely 


associated ideas. 


Leonardo’s insight made a connection between two very different 
things. He saw a relationship between screws and the challenge of flight. 
A propeller amounts to an air screw, holding on to air much as a wood 


screw holds on to wood, albeit less firmly. 


(2000, p. 3) 


Figure 41.1 Leonardo’s aerial screw (Photo: Claudio Divizia / Hemera / 
Thinkstock / Getty Images Plus). 


Kounios and Beeman: Insight and Remote Associations 


Kounios and Beeman (2015) have studied creative thinking mainly through 
the use of compound remote associates (CRA) problems. Mednick (1962) 
used such problems in his Remote Associates Test (RAT). In CRA problems, 
people are given a set of three words and are asked to find a fourth word that 
is linked to them. Given these three words — mouse, blue, cottage — what 


other word is related to all three? The answer is cheese: mouse cheese; blue 


cheese; cottage cheese. All those pairs are only remotely linked: none of the 
other words is a strong associate to cheese. Upon solution, people are asked 
to report if it came in step-by-step form or suddenly, in an insight. One of 
Kounios and Beeman’s discoveries is that the right hemisphere is active when 
people report that they solve such problems through insight. In Kounios and 
Beeman’s view (2015, p. vii), processing in the right hemisphere makes 
available information only remotely associated to the situation. 

Kounios and Beeman (2015) apply this perspective to the analyses of 
numerous creative advances, one of which is “Wag” Dodge’s invention of the 
escape fire. R. Wagner (“Wag”) Dodge was the chief of a crew of 
smokejumpers who parachuted into Mann Gulch, Montana, on August 5, 
1949, to fight a forest fire. Initial reports indicated that the fire was small and 
not very dangerous, but as the men approached it, the winds changed 
direction, toward them, and became much stronger. The fire “blew up.” The 
men turned and ran, but Dodge realized that they could not outrun the fire; he 
thought that they all would die. However, Dodge then had a near-miraculous 
insight: he took a match and lit the grass in front of him, away from the 
oncoming fire. When a patch of grass had been burnt out, he stepped into that 
area and called for his men to join him, but none did. Dodge wet a cloth and 
put it over his face and lay down on the still-smoldering charred ground. The 
fire passed around him, and, although the winds from the fire were so strong 
that he was lifted off the ground several times, he survived. Two 
smokejumpers who were able to reach the top of a ridge also survived the 
fire; 13 others died trying to outrun it. Setting an escape fire is now taught to 
all smokejumpers as part of their training. 

In discussing Dodge’s insight, Kounios and Beeman (2015) emphasized 


the restructuring that took place in his thinking. 


Dodge’s insight was a sudden flip of his understanding. His radical 
reinterpretation was utterly nonobvious: Fire wasn’t just the problem — it 
was also the solution. He fought fire with fire. 


(p. 23; emphasis theirs) 


That restructuring served to make available elements only remotely related to 
the problem Dodge was facing. Dodge’s setting an escape fire, that is, “using 
a fire as a tool for creating a protective buffer zone ... [was] weakly 
associated with elements of the problem” (p. 89). Kouonios and Beeman also 
highlighted Dodge’s “openness to alternative, nonobvious interpretations. 
This is how it broke out of his box” (p. 36). In Kounios and Beeman’s view, 
Dodge’s insight came about only because of his rejection of everything he 


knew about fighting fire. 


Remote Associates and Creativity: Conclusions 


The remote-associates viewpoint starts with the assumption that creativity 
depends on bringing together ideas that have in the past been unrelated. 
Theorists have proposed several mechanisms — unconscious processing, flat 
associative hierarchies, cognitive disinhibition, broad attention, relinquishing 
executive control, restructuring — to forge those linkages. Here I have focused 
on one sort of evidence brought forth to support those assertions: case studies 
of creative advances, because they deal directly with creative advances at the 
highest level, and are, therefore, a potentially important source of support for 
the remote-associates view. We now turn to the expertise analysis of 
creativity, which proposes that creative thinking is based firmly on 
knowledge and a structured imagination. Here too I will examine case studies 


of creative advances. 


Expertise in Creative Thinking 


My emphasis on the positive role of expertise rests on the assumption that the 
world is constantly changing: one never faces the same situation twice. All 
our behavior must, therefore, be adapted to a changing environment, and 
there is thus a creative core to everything we do. Ohlsson (2011) also 
assumes that the world is in flux, but he believes that those changes are on a 
very wide scale, making it impossible to deal with the present on the basis of 
the past (see also Kounios & Beeman, 2015, p. 10). Perkins’s (2000) 
discussion of the world as an “unreasonable” place is based on a similar set 
of assumptions. The expertise view, in contrast, assumes that changes in the 
world are slow: the present is similar enough to the past that attempting to 
transfer one’s expertise is always a reasonable way to try to solve a problem. 
Indeed, if the present were not amenable to analysis based on the past, it is 
hard to understand why humans evolved the capacity to remember it. This 
general conception is not new: psychologists with such disparate orientations 
as Piaget, Skinner, Pavlov, and Hull discussed how organisms respond to the 
present by using the past. Piaget’s assimilation and accommodation (Flavell, 
1963) and the learning theorists’ stimulus and response generalization 
(Bower & Hilgard, 1981) were formulated to deal with those phenomena. 
More recently, expertise has been postulated as playing a similar role in 
cognition (Ericsson & Poole, 2016; Weisberg, 2006a, 2006b). 

Modern emphasis on expertise in creativity can be traced back to de 
Groot’s (1946/1978) seminal study of chess masters’ skills. De Groot 


examined the ability of masters to pick a promising move in the middle of a 


chess game, and he found that they very quickly homed in on a very small 
number of possible moves, spending no time contemplating unpromising 
moves. De Groot concluded that a critical component of masters’ skill 
involved the ability to recognize promising moves, based on a database of 
chess positions from master-level games, acquired through years of study and 
simulated play. Chase and Simon (1973) extended de Groot’s research, 
analyzing that database. They concluded that a chess master might be able to 
recognize 50,000 patterns of chess pieces from master-level games, acquired 
over ten years of study and practice. Hayes (1989) examined whether that 
“ten-year rule” held for creative domains, such as painting, poetry, and 
composition of classical music. He measured the time between an 
individual’s beginning study in those fields and production of a 
“masterwork,” defined as a work cited in reference works or included in 
compendia. In all those fields, long periods of what Hayes called preparation 
intervened between beginning study and production of a masterwork. 
Hayes’s study provided support for the idea that creative production 
depended on long-term immersion in a domain, which provides the 


opportunity for acquisition of expertise. 


Types of Transfer of Expertise in Creative Thinking 


The degree of transfer of one’s expertise to a new situation depends on the 
specificity of the match between that situation and one’s experience. One can 
postulate a scale of “remoteness” for measuring transfer of expertise (Dunbar, 
1995). A familiar problem will result in problem-specific transfer, the recall 
of knowledge directly related to the problem, which serves as the beginning 
of the process of producing something new. Consider a researcher, carrying 
out an experiment on the Zika virus, who runs into a methodological 
difficulty. The researcher might deal with that difficulty by using what she 
knows about another Zika experiment. That situation can be called local 
transfer: the problematic experiment and the possible solution are from the 
domain of Zika studies. 

If no problem-specific expertise is available, one might be able to carry 
out analogical transfer, i.e. transferring to the new problem information from 
an analogically related situation. To continue our example, if no information 
from Zika experiments is available, the researcher might recall an experiment 
on the Ebola virus that could help to solve the problem. In this case, the two 
situations are analogous: both are experiments with viruses. Using that 


information would be an example of regional transfer (Dunbar, 1995), since 


Zika and Ebola are in the same “region” of the researcher’s conceptual space, 
i.e. viruses. Finally, let us say that the researcher was able to use information 
from a study on insight in problem-solving to overcome the difficulty with 
the Zika experiment. In this hypothetical case, the transfer would be remote, 
i.e. from a domain unconnected to the Zika domain. It is useful in this context 


to use the term near transfer of expertise to denote local and regional 


transfer; far transfer, in contrast, denotes remote transfer. I assume that 
creative advances are never based directly on remote transfer: in order for a 
situation to call forth an idea, there must be some link between that situation 
and that idea. This assumption can be contrasted with the remote-associates 
view, which, as we have seen, postulates several sorts of mechanisms — 
unconscious thinking, cognitive disinhibition, restructuring, etc. — that can 


bring together previously unconnected ideas. 


Heuristic Methods 


If presentation of a problem results in no problem-specific or analogical 
transfer, then the individual must fall back on more general methods, i.e. 
heuristics. Heuristics include reasoning, such as logical analysis of the 
demands of the problem; applying arithmetic or computational processes to 
the problem; working backward from the goal to try to determine the solution 
path; and breaking the problem into parts. Strictly speaking, heuristics 
involve expertise (e.g. we are experts in using language-based logical 
reasoning to determine the implications from the description of a problem). 
However, the “directions” in a heuristic are at a general level — e.g. “analyze 
the logical implications of the information in the problem” — and say nothing 
about the specific problem being contemplated. Therefore, it is useful to 
differentiate between heuristics and other types of expertise, in which there is 
a more specific connection to the problem. 

As one example of the use of heuristics in solving problems, Fleck and 
Weisberg (2013) found that solutions to the Lilies problem, an “insight 
problem,” could arise through an individual’s logical analysis of the 


information in the problem. 


Lilies double in area every day. On the first day of the month, there is 
one lily on the lake. On the 30th day of the month, the lake is completely 


covered with lilies. On what day is the lake half-covered? 


Most people on hearing this problem give day 15 as the answer. They are 
then told that that answer is incorrect, which can cause them to reexamine the 
problem, which can lead to the reasoning out of the conclusion that the 
answer is day 29: if the lilies double in area every day, then the day that the 
lake is half-covered is (by deduction) the day before the day that the lake is 
completely covered, that is, day 29. 


Degrees of Transfer of Expertise: Summary 


Solutions to problems can be based on different degrees of specificity of the 
match between one’s expertise and the problem. That match can range from 
problem-specific knowledge, to analogies, to heuristics. As one goes across 
those levels, the match between the problem and one’s expertise becomes 


more general in nature (Fleck & Weisberg, 2013). 


Restructuring: New Information Stimulating New Ideas 


We have so far considered situations in which an individual’s response to a 
new situation is based relatively directly on transfer of expertise. Sometimes, 
however, an individual can in some situation produce an idea very different 
than anything produced before: an architect might make a radical change in 
style, or a scientist might produce a theoretical proposal that involves a 
radical shift in ideas. Those are the kinds of phenomena that seem to support 


the remote-associates perspective (Kounios & Beeman, 2015). Based on the 


transfer of expertise view, such outcomes cannot occur directly, since there is 
no link between the ideas in question. However, radically new ideas might 
arise in two ways (Fleck & Weisberg, 2004, 2013; Weisberg, 2015 and 
forthcoming). First is through new searches based on failure. Attempting to 


transfer expertise to a problem is a dynamic process. If the response based on 
transfer is not successful, that failure provides new information that will then 
serve in a new search of memory. That search might result in a very different 
outcome, perhaps even a restructuring of the situation, because information 
from failure can stimulate the realization that a whole class of methods is 
incorrect and that a different kind of solution method might work. As an 
example, a person attempting to solve the Candle problem (Fleck & 


Weisberg, 2004; see Figure 41.2) might try to tack the candle to the wall, 


only to realize that the tacks are not long enough to hold the candle. That 
realization could stimulate a search for an object to serve as a shelf for the 
candle (i.e. a restructuring of the problem), which could lead to the tack-box. 
Rather than assuming that a state of impasse triggers a restructuring of the 


problem situation, restructuring based on failure assumes that the same 


mechanism is at work all the time — search of expertise based on information 


from the problem — but the search information has changed. 


Wall 


Figure 41.2 The Candle problem. Using the objects in panel A, attach the 
candle to the wall so that it can burn properly. The “box solution” is shown 
in panel B (Source: 


https://commons. wikimedia.org/wiki/File:Genimage.jpg). 


A second mechanism through which a radically new idea can develop is 
as a result of information from the external environment (Seifert, Meyer, 
Davidson, Patalano, & Yaniv, 1995; Weisberg, 1986, 2006b, ch. 5). A person 


can get an idea from another person or through an encounter with an object. 


While carrying out some other activity, a person may “stumble upon” 


information that is relevant to an unsolved problem. This mechanism can 


“shortcircuit” the search mechanism that ordinarily serves as the basis for 


creative advances. 


Transfer of Expertise in Creative Thinking: Summary 


The transfer of expertise analysis proposes that creative thinking works 
through one basic mechanism: retrieval of information based on the match 
between one’s expertise and the situation one is facing. That match can occur 
at varying degrees of specificity, which determines the specificity of the 
transfer that will occur. In addition, new information, arising from a failed 
solution or acquired independently of the problem-solving process, can result 
in retrieval of a new solution type, in a “restructuring” of the situation. The 
critical aspect of the expertise view is that it assumes that all creative thinking 
is based on near transfer, rather than remote associations. Numerous studies 
of seminal creative advances support the analysis just outlined (Weisberg, 
2006b, chs. 1, 5, 2009, 2011). 


Case Studies of Creative Advances: Expertise in Creative Thinking 


The Double Helix 


The discovery of the double helix of DNA by Watson and Crick is one of the 
most significant scientific discoveries of the twentieth century (Weisberg, 
2006b, ch. 1). Watson and Crick were part of a community of researchers 
working on the problem of the structure of DNA, and one reason that Watson 
and Crick were successful, when the others were not, was that, early in their 
collaboration, Watson and Crick made two crucial assumptions about how 
they would approach the problem of the structure of DNA: they would 
assume that DNA was a helix, and they would use molecular modeling to 
construct the structure. Those assumptions enabled Watson and Crick to 
focus on critical pieces of information, which put them ahead of the other 
researchers. In accord with the transfer of expertise analysis, those critical 
assumptions were based on Watson and Crick’s expertise. They built on the 
work of Pauling, who had proposed a single-stranded helical structure for the 
protein alpha-keratin, and had used molecular modeling to determine that 
structure. Proteins and DNA are analogous molecules: both are organic 
macromolecules, constructed from repeating units or building blocks (amino 
acids versus nucleotides), and those building blocks are connected in the 
Same way in each macromolecule (peptide bonds versus phosphate bonds). 
Proteins are polypeptides and DNA is a polynucleotide. Thus, Watson and 
Crick used, as the basis for their strategy, what they knew about research on 
an analogous molecule — a regional analogy. 

Transferring expertise based on Pauling’s success with alpha-keratin 


went only so far, since available data indicated that the DNA molecule was 


thicker than a single-strand helix. Watson and Crick did not know, among 
other things, how many strands were in DNA, or how those multiple strands 
were held together. Given that Watson and Crick were at a loss concerning 
the details of DNA, they used heuristic methods to move forward. As one 
example, they used information about the density of DNA to deduce that the 
molecule was made up of three strands. That deduction led them to develop a 
triple helix, a three-strand molecule, as their first model of DNA. That model 
was shown to colleagues, who strongly criticized aspects of it. That negative 
feedback caused Watson and Crick, again using their expertise, to rethink 
several of the model’s components. In addition, they acquired several critical 
pieces of information from outside sources that pointed to two rather than 
three strands. 

The final step involved determining how the strands were held together. 
Each of the nucleotide building blocks of DNA is made up of a sugar, a 
phosphate, and one of four bases (abbreviated as A, C, G, and T). It was 
thought that the bases played a role in holding the strands together, but the 
specifics were unknown. Because no transfer was possible, Watson used 
another heuristic, that of trial and error, to work out the configuration of the 
bases: he manipulated cardboard models of the nucleotides on his desk until 
he discovered combinations that fit together in the required physical 
configurations (Weisberg, 2006a). 

In conclusion, discovery of the double helix was based on near transfer 
and various heuristic methods. No evidence was seen for the utilization of 
remote associations. Watson and Crick’s expertise was the foundation on 
which they built their innovation. I now tum to a very different domain — 
industrial design — for a case study that supports the same general 


conclusions. 


IDEO’s Shopping Cart 


IDEO is one of the most successful industrial-design consulting firms in the 
world, with numerous products and awards to its credit. A number of years 
ago, they accepted an assignment from the television program Nightline, to 


create a new shopping cart (for details, see Weisberg, 2009). They were given 


a week to develop the new cart. The result of their efforts can be seen on the 
Web (www.ideo.com/post/reimagining-the-shopping-cart), and they created 
something that went significantly beyond the extant products. Several novel 
aspects of IDEO’s cart are listed in column I in Table 41.1. It is a sleek cart 
that is very maneuverable, due to casters front and back. It has plastic baskets 
that can be removed by the shopper so that he or she can leave the cart in a 
crowded aisle and go directly to the shelves. The cart also has a child seat, 
with a bar that keeps the child securely in the seat and also functions as a play 
surface. There is a microphone that allows communication with the store’s 


customer service counter, as well as a scanner to allow self-checkout. 


Table 41.1 Novel components of IDEO’s shopping cart and where they came 


from. 
III. Basis for 
I. Innovation II. Problem/origin innovation 
(1) Sleek design Ugly shape/common Expertise: IDEO’s 
knowledge designers’ 
sensibilities 
(2) Casters: allow Hard to navigate in Near transfer: 


more aisles/from shoppers; also wheels and casters 


maneuverability 


(3) Small modular 
baskets: can be 
removed and 
carried. More items 
transported to cart 


(4) Plastic baskets: 
not fire-resistant 


(5) Safety bar on 
child’s seat: keeps 
child in seat 


(6) Play surface on 
child’s seat in cart: 
provides area for 
child to play 


(7) Microphone in 
cart: contact 
customer service 


(8) Scanner in cart: 
shopper scans items 
and checks out 


common knowledge 


Hard to navigate cart in 
crowded store: Use cart as 
“home base” and bring 
items to cart/from shoppers; 
also common knowledge 


Pilferage: Metal baskets 
serve as barbecues/from cart 
buyer 


Child safety: Unattended 
child leaves safety 
seat/common knowledge 


Child irritability/common 
knowledge 


Difficulty finding items > 
Need to contact customer 
service/common knowledge 


Long lines, slow 
checkout/common 
knowledge 


Logic/Observation 


Logic 


Near transfer: 
“safety seat” on 
roller coaster 


Logic: play reduces 
child’s irritability 


Logic + Near 
transfer 
(cellphone?) + 
IDEO’s “electronic 
gadgets” expertise 


Logic + IDEO’s 
“cyberize” 
expertise 


In discussing IDEO’s philosophy and innovation methods on Nightline, 


the president of the company says that IDEO is where “the crazies live,” 


indicating that IDEO’s people think in unusual ways. However, examination 
of the development of IDEO’s shopping cart indicates that it conforms well 
to the expectations of the expertise perspective. IDEO’s staffers had no 
expertise concerning shopping carts, but, immediately after accepting the 
assignment, they went to a supermarket and talked with the shopping cart 
buyer. They also observed shoppers, some of them professionals, in order to 
see directly any difficulties arising from use of shopping carts. Thus, the 
IDEO team quickly acquired expertise about problems with shopping carts, 
shown in column IT of Table 41.1, which served as focal points for their 
efforts. 

The origins of IDEO’s solutions are shown in column III in Table 41.1. 
The innovative aspects of the shopping cart are manifestations of the team’s 
expertise. Several were the result of near transfer, such as changing the back 
wheels on the cart to casters, which are wheels designed to increase mobility 
of objects in restricted areas. Some of the innovations arose from logical 
analysis of the problems, such as the development of plastic baskets. That 
change was in response to the cart buyer’s telling them that supermarkets had 
problems with pilferage, because people used carts with metal baskets as 
barbecues. Changing to plastic baskets eliminated that use. Still other 
solutions were the result of logic plus the particular “cyber-expertise” of the 
IDEO staffers. Finally, the overall sleek design of the cart arose out of the 
designers’ aesthetic sensibilities, another expression of their expertise. 

In conclusion, IDEO may be where the “crazies” live, but their creative 
process is anything but crazy. IDEO’s shopping cart was built out of the 
processes already discussed; one sees no leaps based on remote associations. 


I now turn to still another domain, architecture, to examine Frank Lloyd 


Wright’s design of Fallingwater, the house over a stream in the woods in 


western Pennsylvania. 


Fallingwater 


Wright’s Fallingwater has been called the most famous private dwelling in 


the world (Weisberg, 2011; see Figure 41.3). On December 18, 1934, Wright 


first visited the location where the house was to be built. Almost 20 years 
later, he described his experience. “There in a beautiful forest was a solid, 
high rock-ledge rising beside a waterfall and the natural thing seemed to be to 
cantilever the house from that rock-bank over the falling water” (quoted in 
Toker, 2003, p. 137). Why was it “natural” for Wright to conceive of 
cantilevering a house over the falls? It was “natural” because he had done it 
before. In the 1920s, Wright designed and built Taliesin, a compound of 
buildings in Wisconsin, where he lived and worked. Wright had built an 
artificial waterfall there, by damming a stream. In order to provide electricity, 
Wright obtained hydroelectric power by means of a small power plant 
cantilevered over the waterfall. So it was indeed natural — based on Wright’s 
expertise and near transfer — to do a similar thing when designing a house in a 


location where a waterfall was the dominant feature. 


Figure 41.3 Frank Lloyd Wright’s Fallingwater. 


© 2016 Frank Lloyd Wright Foundation, Scottsdale, AZ / Artists Rights 
Society (ARS), NY. 


A second question concerns the specific design of the house, and here, 
too, we see Wright’s expertise at work through near transfer (Weisberg, 
2011). The structure of Fallingwater is that of a Prairie House, a design that 
Wright had developed many years earlier (see Figure 41.4). A prototypical 
prairie house has a low or flat overhanging roof; two balconies, or one 
balcony above a terrace; a frieze of glass running across the upper balcony; a 
central stone core, including a fireplace; and a cruciform floor-plan. 
Fallingwater has all those components, demonstrating near transfer in its 
design. Wright himself noted that Fallingwater was not different in any 
significant way from the Prairie House. Fallingwater is thus another example 


of expertise serving as the foundation for creative achievement at the highest 


level. Similar processes were at work in Picasso’s creation of Guernica, his 


great anti-war painting. 


Figure 41.4 The Robie house; a Frank Lloyd Wright Prairie House. 


© 2016 Frank Lloyd Wright Foundation, Scottsdale, AZ / Artists Rights 
Society (ARS), NY. 


Picasso’s Guernica 


Picasso created Guernica during May—June 1937, in response to the bombing 
of Guernica, a town in northern Spain, by the German airforce during the 
Spanish Civil War (Weisberg, 2006), ch. 1). During Picasso’s first few days 
of work, beginning approximately three days after news of the bombing, he 
drew eight sketches of the overall composition. Each of the major characters 
in the painting — the bull, the horse, and the woman holding the light — 
appeared in at least six of the eight sketches, usually in the same positions as 
in the painting. Thus, Picasso had the overall structure in mind from the 
beginning. Where did that idea come from? Picasso used other works, his 
own and other artists,’ as the structure on which he built Guernica. One 
Picasso work, Minotauromachy, created in 1935, contains many of the same 


characters, in the same physical layout, as Guernica, indicating that near 


transfer based played an important role in Picasso’s conceiving the overall 
structure of Guernica. Picasso’s expertise can also be seen in his creation of 
the individual characters in the painting, but due to space limitations that 


cannot be discussed here. 


Pollock’s Poured Paintings 


Let us now consider artistic creativity on a broader scale: the development of 
a radically new style. In the late 1940s, Jackson Pollock produced a series of 
paintings by pouring paint on the canvas or flicking paint onto the canvas 


with a stick (Weisberg, 2006b, ch. 5). The resulting works were comprised of 


complex patterns of intersecting looping lines of paint that covered the whole 
canvas. These works, with their completely new technique and their totally 
non-representational style — a complete absence of recognizable objects — 
burst onto the art world, elevating Pollock to the highest echelons. Where did 
that radically new style of painting come from? 

In the 1930s, under the auspices of the WPA (Works Progress 
Administration), a painters’ workshop was organized in New York, under the 
direction of David Alfaro Siqueiros, a Mexican painter. Siqueiros was one of 
three Mexican painters — Diego Rivera and José Orosco were the others — 
who had established a presence in the art world. The three were communists, 
and their politics influenced their theories about art. They emphasized the 
need to remove art from its bourgeois pedestal and bring it to the people. This 
involved, among other things, using modern materials, such as industrial 
paints; and rejecting the brush for non-traditional methods of applying paint 
to the canvas, such as using airbrushes and throwing and pouring. Siqueiros 


had created at least one painting in which paint was applied to the canvas in 


several different ways, including traditional brushing but also pouring. 
Pollock attended Siqueiros’s workshop, and Pollock and two friends created a 
painting at that time that involved poured paint. So, radical though Pollock’s 
technique seems on first sight, it involved expertise and near transfer. In 
Pollock’s “associative hierarchy,” pouring was a near associate to “How does 
one put paint on canvas?” We now turn to a seminal invention — the Wright 
brothers’ invention of the flying machine — to demonstrate further the role of 


expertise in creative thinking. 


The Wright Brothers’ Control System 


When the Wright brothers began to think about developing a flying machine, 
in the mid-1890s, there were several groups already working toward that goal 


(Weisberg, 2006b, ch. 5). Two things set the Wrights apart. First, the other 


groups assumed that it would be impossible for a human to control an 
airplane once it was in the air, due to the unpredictable nature of the winds. 
They therefore worked on designing automated control systems, which would 
relieve the individual of all responsibilities. The Wrights, in contrast, 
assumed that a pilot had to be in control, or else the plane could not succeed. 
In addition, they believed that a human could control a plane in the air. Based 
on that belief, they developed a control system, in which the tips of the wings 
of their aircraft were tilted up and down, a movement they called “wing 
warping,” which allowed the pilot to control the plane. Three questions arise 
here. Where did the Wrights’ belief in the necessity of a control system come 
from? Where did the belief that a human could control the plane come from? 


Where did they get the idea for warping the wings? 


The need for a control system was obvious, at least to the Wrights. Otto 
Lilienthal, a German engineer, had in the 1880s and 1890s made numerous 
gliding flights, gathering skill and information that he could use in designing 
a powered flying machine, and becoming a minor celebrity in the process. 
Lilienthal flew his glider while hanging from a bar under the wings, and 
controlled it by shifting his body, which was problematic as a control 
mechanism, especially in gusts of wind. In 1896, he lost control and crashed 
during a glide, which broke his back and killed him. Lilienthal’s death was 
widely reported in newspapers worldwide, and the Wrights were aware of it. 
They took Lilienthal’s death very seriously, which was one of the reasons for 
their belief in the need for control. It is also interesting that the other groups 
working toward inventing a flying machine were directed by older men — e.g. 
Octave Chanute and Samuel Langley — whose young assistants would have 
the assignment of riding on the plane during test flights. Not having to 
actually ride on a potentially uncontrollable flying machine, with possible 
fatal consequences, might have insulated Chanute and Langley from the fear 
arising from the real dangers that flight posed for humans. The Wrights 
planned to fly their plane themselves, so the potential danger, and the need 
for a control system to deal with it, were of direct importance to them. 

Concerning the belief that a human could control a plane in flight, the 
Wrights had familiarity with a dynamic control system for a moving vehicle, 
through their expertise with bicycles, which had become an obsession in the 
United States in the 1880s. The Wrights had designed, built, and sold 
bicycles. Riding a bicycle requires control very similar to that involved in 
flying an airplane. Turning a bicycle is a dynamic operation, in which one 
leans into the turn and keeps from falling by maintaining speed. Turning on a 


bicycle involves a controlled fall, similar to what happens when an airplane 


banks into a turn. Since the rider of a bicycle can maintain control through a 
turn, a pilot should be able to do the same in an airplane. James Means, who 
wrote several books on the progress in the development of an airplane, had 
predicted that the airplane would be invented by “bicycle men,” because 
flying was like “wheeling” (Weisberg, 2006b, ch. 5). So the Wrights’ belief 


in the possibility that a human could control a plane in flight was the result of 


expertise and near transfer. 

Once the Wrights had decided on the need for a control system, they 
searched for information concerning how to design one. They got that 
information from observing “natural flying machines,” i.e. birds. They 
discovered through observation that birds while gliding control their 
orientation through opposing movements of the tips of their wings: to tilt one 
way, one tip is raised while the other lowered; the opposite movements are 
carried out to tilt the other way. That information led to the idea for wing- 
warping. So the Wright brothers also used near transfer in developing their 
control system. In addition, the Wrights planned to train themselves in the 
use of their control system using smaller-scale gliders, so that they would 


become experts in using any system that they developed. 


Transfer of Expertise and Structured Imagination in Creativity: 
Implications 
The case studies just outlined are consistent in demonstrating the importance 
in creative thinking of near transfer based on expertise. Those conclusions 
conflict with those arising from the case studies presented earlier in support 
of the remote-associates view — Wilkins’s invention of radar; Edison’s light 
bulb; Leonardo’s aerial screw; and Dodge’s escape fire — in which remote 
associations were assumed to be critical. I will now re-examine those cases, 
to demonstrate that, in contrast to the claims of momentous acts of inspiration 
and far-ranging leaps of analogy, in each case the creative advance was based 
on expertise: there was in place a conceptual structure that provided the 


scaffolding on which the advance was built. Near transfer was critical. 


Re-Examination of Case Studies Presented in Support for the 
Remote-Associates View 
The expertise view assumes that all creative thinking depends on near — or 
already-established — associative connections. We should never find an idea 
coming about as the result of the bringing together of two ideas that have no 
link between them. Let us now apply this perspective to the case studies 


purported to support the remote-associates view. 


Wilkins and Radar 


In presenting Wilkins’s invention, Ohlsson (2011) described Wilkins’s 
“momentous act of inspiration” (p. 53): there was the request from Wilkins’s 
superior and then there was radar. However, the situation involved more than 


a single unprecedented leap of inspiration (Weisberg, 2015). Wilkins was 


familiar with a recent government report discussing the problem of airplanes 
interfering with radio waves, which is the basic idea on which radar is built 
(Buderi, 1996, p. 5D: see also 
www.purbeckradar.org.uk/biography/wilkins arnold.htm). So there was a 
link between the problem set to Wilkins and his invention. His superior’s 
request might have retrieved the information that aircraft interfered with radio 
waves (the request contained both “radio waves” and “aircraft”). Wilkins 
might then have inferred that such interference could provide information 
about an aircraft’s location, which led to radar. Thus, Wilkins’s advance 
involved a sequence of steps, building on his expertise, rather than one 


“momentous” leap, independent of everything he knew. 


Edison’s Light Bulb 


Edison’s invention of the carbon burner for his light bulb can be analyzed 


similarly (Weisberg, 2015). Edison was not the first to attempt to invent a 


light bulb. There had been several unsuccessful attempts, most of which had 
used either carbon or platinum as possible bumers. Edison knew about at 
least some of those unsuccessful attempts, so the reason why he concentrated 
on platinum and carbon was because of near transfer. In addition, Edison had 
used chemistry reference works to determine that platinum and related 


metals, with high melting points and resistance to oxidation, might be good 


candidates for a burner. Ohlsson (2011, p. 134) described Edison’s switching 
from platinum to carbon as a sudden restructuring of the situation, and a 
move in a totally different direction. However, in addition to being familiar 
with others’ work using carbon, Edison had started his research with carbon, 
over a year earlier (Friedel, Israel, & Finn, 1986, p. 16; Israel, 1998, p. 171). 


In order for a carbon burner to work, it had to be in a vacuum; otherwise it 


would burn up (oxidize). However, the available vacuum pumps were not 
efficient enough to produce the required near-complete vacuum, so the 
carbon oxidized and the light failed. Edison then turned from carbon to the 
other option, platinum, that he knew about from others’ research. 
Furthermore, when Edison switched back to carbon and found success, it was 
because his staff had developed a very efficient vacuum pump (Freidel et al., 
1986), which solved the critical problem with carbon. The design for this 
pump came from a journal article. It thus seems that the return to carbon was 
not the result of a restructuring in response to an impasse; rather, it was a 


return to a previously tried material because circumstances had changed. 


Once again we can provide a structure — based on Edison’s expertise and that 


of his staff — to serve as a scaffold for creative thought. 


Leonardo’s Aerial Screw 


Leonardo’s aerial screw was based on an analogy between the wood screw 
and the sail on his flying machine, and Perkins (2000) emphasized how 
different the two concepts were. However, here, too, there was a link in 


Leonardo’s expertise between screws and flight (Weisberg, 2015). In his 


scientific studies, Leonardo had analyzed the characteristics of air, one of the 
four basic elements in Renaissance science. He had discovered that air, 


although invisible, was compressible (Laurenza, 2006, p. 47). Air was like 


wood, which was compressible, but different from water, which was not. The 
compressibility of air was the link that could have led Leonardo to the 
possibility that a screw-shaped device, if rotated quickly enough, could bore 
through the air and lift itself in flight, just as a rotating screw is drawn up into 
a ceiling beam. Leonardo had a long-standing interest in human flight, so the 
compressibility of air and wood might have been particularly noteworthy to 
him. Furthermore, in his notebooks describing birds’ soaring flight (i.e. the 
bird rides the rising currents of air without flapping its wings), Leonardo 
described those flight paths as sometimes being spiral, “in the manner of a 


screw” (Richter, 1952, p. 97), which might have provided another link 


between wood screws and the aerial screw. In conclusion, Leonardo’s 
making a connection between screws and flight was not the far-reaching leap 


that Perkins assumed that it was: it was an extension of his expertise. 


Wag Dodge’s Escape Fire 


Kounios and Beeman (2015) described Dodge’s insight as being “utterly 
nonobvious: Fire wasn’t just the problem — it was also the solution. He fought 
fire with fire” (p. 23; emphasis theirs). They also emphasized that “using fire 
as a tool for creating a protective buffer zone [is] weakly associated with the 
elements of the problem” (p. 89). However, questions can be raised about 
both those statements: Dodge knew how to fight fire with fire, and he also 
knew about using fire to create a protective buffer zone (Weisberg, 2013). 

Smokejumpers’ training included building a backfire, a fire that moves 
back toward the main fire, consuming the fuel that the main fire needs to 
continue advancing. In other words, building a backfire entails both fighting 
fire with fire and creating a buffer zone. Smokejumpers were also taught that 
they might be able to turn into the fire and try to find burnt-out spots in which 
you might be safe, i.e. finding an existing buffer zone. Those ideas — fighting 
a fire with fire; creating a buffer zone; and finding an existing buffer zone — 
were part of smokejumpers’ expertise, and were near associates to the 
problem of dealing with that fire. 

The Mann Gulch fire was moving too fast to allow Dodge to build a 
backfire, and the fire’s front wall was several hundred feet deep, making it 
impossible to find burnt-out spots. However, Dodge’s invention can be seen 
as a combination of those methods. Instead of finding burnt-out space by 
walking into the advancing fire, or producing it by setting a backfire (i.e. a 
burnt-out space between him and the fire), he produced it by setting his 
escape fire. That is, Dodge produced burnt-out space in the direction he and 


the fire were moving. To understand Dodge’s insight, we do not need to 


postulate remote associations on his part. Contrary to Kounios and Beeman’s 
analysis, near transfer based on expertise seems to have been critical in 


Dodge’s insight. 


Case Studies Reconsidered: Near versus Remote Associations 


This analysis indicates that the case studies brought forward in support of the 
remote-associates view can be understood as resulting from expertise, which 
in each case provided a structure supporting the thinking that led from the 
problem to the proposed innovation. This is a particularly strong conclusion, 
because those case studies were presented as support for the remote- 
associates view. One potential weakness of case studies is that of selectivity: 
one might choose cases that support one’s view and ignore those that 
contradict it. These cases are thus particularly cogent, because they were 
presented by theorists in opposition to the expertise view. The fact that they 
turn out to support expertise is therefore doubly important. A second issue 
concerning case studies is that their usefulness obviously depends on the 
information available. In that context, it should be emphasized that we are 
dealing here with empirical evidence. For example, it is a fact that Leonardo 
knew that air was compressible; it is a fact that Edison began his work with 
carbon burners; and it is a fact that Dodge knew about backfires. One cannot 
ignore those facts in presenting a case study. Interpreting the facts is, of 
course, a different question, but having certain facts available limits the range 
of possible interpretations. In my view, the facts available about each of those 


case studies call into question the remote-associates interpretation. 


Expertise in Creative Thinking: Some 
Implications and Remaining Questions 


Let us conclude for the sake of discussion that there is support for the 
expertise perspective in the case studies presented here and elsewhere 
(Weisberg, 2006b), as well as in recent laboratory studies of problem-solving 
(Fleck & Weisberg, 2004, 2013; Weisberg, 2015 and forthcoming). That 


conclusion leaves some questions remaining. 


Expertise, Creative Thinking, and Deliberate Practice 


One foundational assumption of the expertise view (e.g. Ericsson & Pool, 
2016, ch. 4) is that deliberate practice is critical in the development of 
expertise. Deliberate practice is a special activity, in which a teacher pushes 
the learner beyond the comfort zone, by concentrating on aspects of 
performance that need improvement. The learner at some point becomes 
capable of directing his or her own practice. It is not clear that the same 
process operates vis-a-vis creative thinking. For example, an individual may 
learn to paint, or to perform or compose music, through deliberate practice, 
but that training does not, per se, involve creative thinking. Picasso’s early 
training as an artist, initiated by his father when Picasso was a child, did not 
provide anything that led directly to the invention of Cubism almost 30 years 
later. So, while it seems that expertise plays a critical role in creative 
thinking, exactly how that plays out in specific creative advances needs more 


fleshing out. 


Remote Associates and Creative Thinking: A Re-examination? 


If we assume that expertise plays an important positive role in creative 
thinking, that conclusion leaves us with a paradoxical situation, because there 
is also support available for the remote-associates view. One example is the 
finding reported by Carson (2014) of a correlation between the amount of 
cognitive disinhibition exhibited by an individual in a laboratory setting and 
the degree of real-world creative achievement that they report. Also, having 
people carry out perceptual tasks requiring broad attentional focus positively 
affects their performance on tasks involving creative thinking (Friedman et 


al., 2003; Wegbreit et al., 2012). One possible response to this paradox is to 


assume that the contradiction is illusory, and that each of those views might 
be relevant to a limited domain. Perhaps transfer of knowledge underlies 
large-scale creative advances and the solutions to the sorts of “insight” 
problems investigated by Fleck and Weisberg (2004, 2013). Solutions to 
other types of laboratory problems, including perhaps CRA problems, may 
come about through utilization of remote associates. 

Another possibility, however, is that the laboratory studies presented in 
support of the remote-associates view are open to other interpretations, which 
might bring them more into alignment with the expertise view. For example, 
Chan and Schunn (Chan, Dow, & Schunn, 2015; Chan & Schunn, 2015), 


have carried out several studies that have examined the effects of near versus 


far analogies on problem-solving. Results indicated that individuals did not 
spontaneously use far analogies in solving problems, and, furthermore, 
providing far analogies did not facilitate creative thinking. So a question can 


be raised about a critical assumption of the remote-associates view. These 


findings fit with those from the case studies reported here. Similarly, 
Cranford and Moss (2012) examined the fine-grained processes underlying 
solution of CRA problems, by collecting verbal protocols, and they found 
that those problems are not always solved as the remote-associates view 
assumes. People might report solving a problem through insight, but the 
protocol indicated that they neither reached impasse nor restructured the 
problem. People reported insight when the solution was the first word that 
came to mind. Such results indicate that support for the remote-associates 
view may not be as broad as it has seemed. 

In addition, Benedek and Neubauer (2013) tested Mednick’s (1962) 
prediction that more- versus less-creative people differ in the steepness of 
their response hierarchies and did not find the predicted differences in 
structure. The response hierarchies of both groups were very similar, 
although the high-creative individuals produced more low-frequency word- 
association responses. Benedek and Neubauer concluded that the difference 
between high- and low-creative individuals might be in the way in which 
they searched their associations, i.e. a difference in executive functioning, not 
in the structure of their associations. Along similar lines, Chein and Weisberg 
(2014; Chein, Weisberg, Streeter, & Kwok, 2010) have reported that 


executive functioning plays a positive role in solving problems with insight. 


Expertise and Creative Thinking: Conclusions 


The remote-associates perspective on creative thinking is pervasive in 
modem psychology and also in our society. The notion that creativity 
requires thinking outside of the box is the remote-associates view turned into 
a truism. However, as this chapter has demonstrated, it may be time to re- 
examine the remote-associates perspective with a critical attitude, because, if 
a theory of creative thinking should be able to deal with creative 
achievements at the highest level, there is reason to believe that the remote- 
associates view is basically misdirected. Creative thinking seems much more 
straightforward than the remote-associates view assumes: creative advances 
develop out of an individual’s attempts to apply his or her expertise to the 
situation that they are facing. Even the most radically new advances begin 
with that process, and far-reaching advances come about through a dynamic 
iteration of that same process. As people work through trying to apply their 
expertise to novel situations, failures result in new information becoming 
available, which results in new directions in thought. One does not need to 
postulate special mechanisms — unconscious thinking, flat associative 
hierarchies, cognitive disinhibition, diffuse attention, restructuring — 
dedicated to the production of new ideas. The mechanism of applying one’s 
expertise to the problematic situation one is facing, through repeated search 
of memory in response to ever-changing information from the environment, 
that is, reiterative thinking inside the box, is sufficient to explain even the 


most radical creative advances. 
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The scientific study of individual differences and the experimental 
investigation of human performance in normal adults portray a pessimistic 
picture of adult development. Ubiquitous findings of negative age-graded 
changes in psychometric ability factors and reduced speed or accuracy in 
most cognitive-motor tasks from the decade of the 30s onward have 
motivated theories of broad decline, such as general age-related slowing and 
general decline in fluid abilities (Salthouse, 2010). Hence, accomplishments 
by older experts and the high performance levels in many older professionals 
present a puzzle. The central questions are whether older experts are 
exempted from general age-related declines and how they can maintain their 
performances into older age. We note that there is no general consensus on 
what constitutes “old” or “older,” terms that depend on socio-historical 


norms. For instance, in US work environments, age 40 might be considered 


“older” based on age discrimination in employment legislation. Age 67, full 
pension age in the US today, might be considered a marker of “old,” whereas 
age 70, pension age in Bismarck’s Prussia, defined “old” a few centuries 
earlier. 

We start out with sketches of the scientific concepts of ability, expertise, 
and their relationship to adult development. We then address age-related 
changes in cognitive processes and their neural substrates and their relations 
to the different theoretical accounts for expert performance in later adulthood. 
Our subsequent review of age-comparative studies in different domains 
makes a strong case for the dominance of acquired skills and mechanisms in 
expert performance. We then address recent studies that attempt to identify 
the neural substrates of expert mechanisms and how younger and older 
experts’ use of brain regions differs from novices. In the final sections, we 
focus on age-related constraints for purposeful and deliberate practice (see 


Ericsson & Pool, 2016, ch. 1, for definitions) in maintaining expertise. 


Abilities and Their Relation to Expertise in 
Developmental Contexts 


Common-sense notions typically attribute high achievement to innate, stable 
dispositions, a view which has its scientific origin in the nineteenth-century 
writings of Sir Francis Galton (1979). Although Galton equally emphasized 
zeal and the power to work hard as prerequisites of outstanding 
accomplishments, later research mainly focused on stable abilities. Two 
major lines of theorizing emerged in twentieth-century research, which can 
be distinguished according to the degree of domain-specificity attributed to 
hypothetical dispositions. The concept of intelligence denotes stable, 
individual differences in domain-general abilities assumed to be relevant to 
learning and acquiring any type of skill. Psychometric tests of intellectual 
abilities indeed proved to be valid correlates of academic achievement 
(specifically, high-school grades), job training, and initial job performance 


(Schmidt & Hunter, 1998). Studies in behavioral genetics have established 


that the abilities measured by these tests have a considerable heritability 
component, about 50 percent for general intelligence, the g-factor (Plomin, 
1990). At the same time, correlations between intellectual abilities and initial 
job performance were found to decline systematically with increasing interval 
from the end of formal training such that psychometric ability measures 
showed only weak to moderate correlations with performance in practicing 


professionals (Hulin, Henry, & Noon, 1990). 


The second line of theorizing associates the notion of innate dispositions 


with highly domain-specific abilities (talents) as causes for extraordinary 


achievements. Related notions emerged in areas such as music and the arts. 
Accordingly, these talents require minimal external stimulation to emerge in 
individuals with the appropriate dispositions (Winner, 1982, 1996). The 
talent view fared less well in scientific contexts because long-term 
predictability of talent tests is weak and professional musicians showed 
remarkably poor performance on tests of musical talent (Howe, Davidson, & 
Sloboda, 1999; Sloboda, 1991). 


In the context of adult aging, long-term performance and the validity of 


abilities for real-life expertise become key issues. The increasing relevance of 
domain-specific knowledge and skills over general abilities has also been 
demonstrated for older professionals, such as bank managers (Colonia- 
Willner, 1998; Wagner & Sternberg, 1991). Experimental studies suggest that 
the role of domain-general abilities for specific skills is itself a function of 
level of accomplishment. Laboratory training studies found general 
intelligence (g) to be a critical factor at early stages of skill acquisition, while 
later stages and performance after practice tends to correlate less with g, but 
increasingly with factors closer to the skill under investigation (Ackerman, 
1988). At a more general level, the contribution of specific over general 
psychometric abilities also appears to depend on individuals’ overall level of 
cognitive functioning with g having its strongest expression in lower scoring 
participants (Detterman & Daniel, 1989; Wai, 2014). 

Taken together these findings demonstrated a reliable impact of 
psychometric intelligence at early stages of skill acquisition and in 
individuals performing at relatively low levels of competence, consistent with 
Cattell’s classic investment theory (Cattell, 1971, 1987). This theory posits 
that primary mental abilities of the fluid (Gf) type like processing speed or 


spatial abilities are invested as building blocks into the development of more 


complex abilities (see also Krampe & Baltes, 2003). Across development, 
levels of experience and other age-correlated factors attenuate the influence 
of those ability factors relevant at earlier stages of skill acquisition. 
Consistent with this proposal Li et al. (2004) found that the correlation 
between processing speed and crystallized intelligence was high (~ 0.50) for 
children aged 6-11 years, but dropped for older age groups reaching its 
lowest value (0.10) for adults in their 30s to 50s. Crystallized intelligence 
refers to skills and knowledge that rely heavily on experience stored in long- 
term memory, making it more similar to concepts of expertise than Gf. After 
middle adulthood, the correlation in the Li et al. (2004) study increased again 
to intermediate levels (0.30) suggesting that fluid mental abilities might yet 
become more important as we age. The psychometric study of abilities and 
their relevance for complex skills provided a starting point for the systematic 
investigation of the processing mechanisms that underlie expert performance. 
We will return to the role of domain-general functions and specific abilities 
for expert performance in later adulthood after discussing the concept of 


expertise. 


Expertise 


Different from the domain-general abilities emphasized by intelligence 
research the concept of expertise refers to individuals’ superior levels of 
performance in specific domains. Theory posits that experts have developed 
specific mechanisms to circumvent the processing limitations constraining 
normal (i.e. novice) performance (Chase & Ericsson, 1981; Ericsson, 
Krampe, & Tesch-Rémer, 1993). This hypothesis not only implies that 
experts do things differently than novices at the level of cognitive processes, 
but also that the neural substrates of expert and “novice” mechanisms differ. 
Acquiring expertise amounts to a gradual decoupling of domain-specific 
expert mechanisms from the domain-general mechanisms underlying 
intelligence abilities. Examples like chess players’ superior move selection 
skills, the advance coordination of fingers in typewriting, and pianists’ hand- 


independence in rhythmic timing (Gentner, 1987; Krampe, Kliegl, Mayr, 


Engbert, & Vorberg, 2000) illustrate that expert mechanisms are not tricks- 
of-the-trade easily learned or taught, but require years or decades for the 
expert to maximally adapt his or her performance to the performance 
constraints of a specific domain. Ericsson and colleagues (Ericsson et al., 
1993) have depicted this development as the long-distance race of deliberate 
practice. They emphasize that effort, motivation, and resource constraints 
determine whether an individual will eventually succeed. These constraints 
continue to be important for the maintenance of expert performance; 
however, aging will naturally change how they exert their influences. For 


instance, maintenance practice might enable older experts to keep performing 


at high levels, but only if the dose is greater than that required for 
maintaining the same performance at a younger age. That is, whereas decline 
through disuse might be countered in professional musicians by playing 
scales daily when young, decline might be driven by both disuse and 
worsening arthritis when old. 

While authors agree that purposeful and deliberate practice plays a key 
role for expertise, debates continue as to whether it is a necessary or 
sufficient factor in attaining expert status (Campitelli & Gobet, 2011; 
Ericsson, 2014, 2016; Macnamara, Moreau, & Hambrick, 2016). A related 
debate is whether primary or innate abilities continue to determine expertise 
development at higher levels of performance. These debates point to the 
importance of understanding mechanisms underlying age-related changes in 


domain-general functions, particularly brain structures and functions. 


General Processing Speed, Intellectual 
Abilities, and the Aging Brain 


The dominant finding in cognitive aging research with normal adults is that 
accuracy of memory processes and speed of most types of perceptual- 
cognitive-motor performance undergo systematic age-related declines from 
young to older adulthood. Adults by their seventh decade of life typically 
need about 1.6 to 2 times as long to process the same tasks as in their 20s 
(Jastrzembski & Chamess, 2007). Large-scale cross-sectional studies 
consistently reveal considerable age-graded declines in performance IQ (e.g. 
perceptual-motor speed, timed reasoning tasks) during adulthood, starting as 


early as age 30 (Kaufman, 2001). Due to the age-referenced definition of IQ 


this means that the average 50-year-old can attain a certain IQ score with 
only 85-90 percent of the performance speed typical of an individual in her 
early 20s. Meta-analytic reviews document age-related declines in working 
memory capacity and reasoning under time pressure, in the ability to perform 


concurrent tasks, and in cognitive control (Verhaeghen, 2014). Cognitive 


control (or executive control) comprises the planning of complex actions, the 
maintenance of task-relevant information (task-sets), monitoring, the 
inhibition of irrelevant information, and the switching of task-sets, when 
performance conditions change. Similar findings have been reported in the 


domain of fine-motor control and movement production (Krampe, 2002). 


Generally, negative age effects tended to be more pronounced if tasks 
require more complex processing like recall vs. recognition or unimanual vs. 


bimanual movement coordination. Cognitive changes correspond to extensive 


structural, functional, and neurochemical brain changes (Hanggi et al., 2015). 


Age-related loss of functionality differs considerably between regions, 
showing an anterior—posterior gradient with prefrontal regions being the first 
and more severely affected (Raz et al., 2004). Different authors emphasize 
the role of prefrontal regions and frontoparietal circuitry for cognitive 
control, attention, and working memory and their age-related changes 
(Campbell, Grady, Ng, & Hasher, 2012; West, 1996). Differential decline in 


brain regions may explain why general abilities relying on frontoparietal 


circuitry are more compromised with increasing age than specific highly 
practiced skills relying on less compromised areas. This expertise advantage 
is most evident in memory-based skills as neuropsychological studies show 
(see below) that expert knowledge is not stored in prefrontal regions. At a 
more general level we may assume that specific, highly exercised regions are 
simply better maintained with age. 

The psychometric tradition considers the processes described above as 
facets of fluid intelligence. Negative age-related changes begin to emerge as 
early as the late 20s. For “crystallized” or “pragmatic” aspects of intelligence, 
which relate to knowledge and experience-based aspects of skill, onsets are 
later, rates of change much slower, and the level maintained into late 


adulthood is much higher than in fluid intelligence (Kaufman, 2001; Li et al., 


2004). Some domains of cognitive functioning appear to be less affected by 
aging than others, presumably because of the compensatory effects of 
accumulated knowledge, for instance, in tasks requiring lexical decisions 
(Lima, Hale, & Myerson, 1991). 

The ubiquity of negative age effects in speeded performance has 
nurtured the development of general factor accounts such as models of 


general age-related slowing (Cerella, 1990), the processing-speed mediation 


of adult age differences in cognition (Salthouse, 1996), or the information- 


loss model of age-related slowing (Myerson, Hale, Wagstaff, Poon, & Smith, 
1990). These models share the proposition that “normal” aging is general in 
that it uniformly reduces the speed and efficiency of cognitive, perceptual, 
and psychomotor functions. We can thus rephrase our initial question related 
to the age-expertise puzzle by asking whether performance in older experts is 
exempted from general age-related slowing, or to put it concisely: “Does it all 


go together, when it goes?” (Rabbitt, 1993b). 


Implications for Non-Laboratory Performance 


From the assumption that these processes form the building blocks of expert 
performance, one would expect such age-related reductions to affect 
professional competence, a form of skilled performance. However, meta- 


analyses found near-zero (McEvoy & Cascio, 1989) or even slightly positive 


(Waldman & Avolio, 1986) relationships between age and productivity in 


work settings. Using a larger database and a wider age range, Sturman (2003) 
argued for an inverted U-shaped function with gains at young ages and 
negative age—performance relations for ages above 49. The overall variance 
related to age in Sturman’s meta-analysis was quite low (3 percent); however, 
these meta-analyses found that age had a more positive relation with 
performance in more skilled employees. Probably the most comprehensive 
meta-analysis was provided by Ng and Feldman (2008). These authors found 
core-task job performance and creativity to be largely unrelated to age, 
although they also observed a U-shaped relation in their data with younger 
workers (age < 40 years) improving and slightly negative relations within the 
subgroup of “older” workers. On productivity indicators other than the core 
tasks, like absenteeism, citizenship- and safety-related behaviors, Ng and 
Feldman even found better performance in older employees. Taken together 
the evidence from organizational psychology portrays a much more 
optimistic picture than lab-based cognitive aging research. There are 
important caveats. First, most studies don’t focus on expert workers. Second, 
studies focus on a relatively “young” older age group (i.e. age 40+), very 
different from usual aging research. Finally, the bulk of meta-analytic data 


come from cross-sectional studies, hence, underperforming employees may 


be systematically dropping from samples, meaning that selection effects may 
mask age declines. 

The evidence from experimental and psychometric research does leave 
the possibility that knowledge and experience can compensate for age-related 
declines in knowledge-rich domains like chess or medical diagnosis. Neural 
net simulation work (Mireles & Charmess, 2002) provides a biologically 
oriented explanation about how acquired structured knowledge may even 
protect working memory function (e.g. long-term working memory, Ericsson 
and Kintsch, 1995) from expected age-related changes in neural network 
integrity that govern learning rate, forgetting rate, and quality of signal—noise 
ratio. However, maintained levels of professional skill or expertise in 
domains with extreme demands on speed and accuracy like air traffic control, 
piloting, or virtuoso musical performance pose a more difficult explanatory 


problem that we turn to next. 


Theoretical Accounts of Expert Performance 
in Older Age 


The biggest challenge for models of expertise and aging is to reconcile the 
observed age-related declines in basic abilities in normal adults with the 
evidence for superior performances in older experts and professionals. A 
second issue relates to the specificity of skills acquired and transfer to other 
skills. Finally, does expertise in late adulthood depend on experts’ 
Maintenance activities or can it be attributed to the acquisition phase? 
Depending on the answers to these questions we can distinguish four 
theoretical accounts in the literature (for related discussions see Charness & 
Bosman, 1990; Krampe & Baltes, 2003; Salthouse, 1991). 


The first account maintains that older experts have undergone decline, 
but from an initially higher baseline, so remain superior in skill-relevant 
abilities that existed prior to expertise acquisition. Such explanations have 
been termed preserved differentiation or a priori disposition accounts in the 
literature. The second position, the expertise-driven general abilities account, 
emphasizes transfer. Accordingly, the process of expertise acquisition 
involves gradual improvements in those abilities that constrain normal 
performance (like working memory span) such that expertise should transfer 
to some (but not necessarily all) broader cognitive functions. The third type 
of account, differential preservation or selective maintenance, posits that 
outstanding performance rests on specific mechanisms that enable experts to 


circumvent the process limitations constraining performance in normal 


individuals (Chase & Ericsson, 1982). Finally, the compensation account 


assumes that older experts actively acquire new mechanisms when 
experiencing age-related decline in mechanisms which supported their 
expertise at younger ages. A critical assumption of this fourth account is that 
the sets of expert mechanisms differ between younger and older experts. 

Variations and combinations of the four accounts are feasible and have 
been proposed. For example, one interpretation of Galton’s original proposal 
is that innate domain-general talents (endurance, ability to focus) promote the 
acquisition of expertise-specific skills. Differential preservation of specific 
skills has been attributed to their special representation in long-term memory 
as the result of intense acquisition processes like overlearning (Bahrick & 
Hall, 1991). Other accounts of the same type emphasize that skills deteriorate 
unless they are actively maintained. 


According to the deliberate practice model (Ericsson & Charness, 1994; 


Ericsson et al., 1993; Ericsson & Lehmann, 1996) expert mechanisms must 
be acquired through individual efforts directed at the long-term adaptation to 
internal (e.g. age-related changes in cognitive functions) and external (e.g. 
task-domain, professional environment) constraints. When applied to aging 
and expertise, the deliberate practice account implies that older experts must 
actively maintain those specific mechanisms that are vital to their domain, 
and we refer to this set of assumptions as the “maintenance through 


deliberate practice” account (Charness, Krampe, & Mayr, 1996; Krampe & 


Ericsson, 1996). This position maintains that expertise in later adulthood is 
not merely the outcome of achievements during younger ages or 
“experience” through extended usage. Rather, older experts must 
continuously invest deliberate effort into the development of their skills while 


adapting to the constraints imposed by aging. Naturally, purposeful and 


deliberate practice may also play a role if older experts acquire specific 


mechanisms to compensate for age-related deterioration in critical skills. 


Cognitive Abilities, Age, and Expertise: 
Empirical Evidence 


Experimental studies with individuals differing in their ages and levels of 
expertise have been conducted in a variety of domains from games, 
professional, and artistic skills. Studies on age and expertise in games 
included chess (Charness, 1981a, 1981b; Charess et al., 1996), bridge 
(Charness, 1989), Go (Masunaga & Horn, 2001), Mastermind (Maylor, 
1994), and crossword-puzzle solving (Rabbitt, 1993a). Age-comparative 


studies of professional expertise comprise typing (Bosman, 1993; Salthouse, 


1984), piloting (Kennedy, Taylor, Noda, Yesavage, & Lazzeroni, 2015; 
Morrow et al., 2003; Taylor et al., 2011; Taylor, Kennedy, Noda, & 


Yesavage, 2007), air traffic control (Nunes & Kramer, 2009), architecture 


and graphic design (Lindenberger, Kliegl, & Baltes, 1992; Salthouse, 


Babcock, Skovronek, Mitchell, & Palmon, 1990), visual search in medical 


assistants (Hoyer & Ingolfsdottir, 2003), fine-motor control in mechanics 
(Vieluf, Godde, Reuter, & Voelcker-Rehage, 2013), accounting (Castel, 
2007), and management skills (Colonia-Willner, 1998; Walsh & Hershey, 
93). Auditory processing (Parbery-Clark, Anderson, Hittner, & Kraus, 


-_ 


2012; Zendel & Alain, 2014), musical memory and perception (Meinz, 


2000), and performance (Krampe & Ericsson, 1996) have been studied in 


young and older musicians. 
The general picture emerging from these studies is that older experts 
show reduced, if any, age-related declines in the efficiencies or the speed at 


which they perform skill-related tasks. At the same time older experts 


showed “normal” (i.e. similar to non-expert controls) age-graded declines in 
general measures of processing speed and cognitive abilities as measured 
through psychometric tests, and performance on unfamiliar materials (see, 
however, Lindenberger et al., 1992; Salthouse et al., 1990). Thus, the 
evidence from age-comparative expertise studies supports the proposition 
that expert performance at any age relies more on specific rather than general 
cognitive mechanisms. Consequently, models of expertise have departed 
from the assumption that the same set of abilities that underlie performance in 
psychometric intelligence tests can also account for the ultimate level of 
expertise attained or the level of expertise maintained in later adulthood. 
Findings from laboratory research on older experts generally correspond 
with the dominant finding in occupational psychology, that age and skilled 
performance are weakly correlated. The likely reason for the age-graded 
stability of performance in older experts is that increased age brings with it 
increased job-specific knowledge and skills. However, there are important 
caveats. These include weaknesses such as restricted age ranges, restricted 
job types, and uncertain reliability and validity in job productivity measures 
(Salthouse & Maurer, 1996). Also, older experts in these studies could 
represent the survivors of an age-graded winnowing process by which 
individuals with stronger age-related declines in relevant capacities, or those 
who have been less motivated to continuously invest in the development of 
their skills, have dropped from their fields of expertise or have transferred to 


less challenging positions. 


Neuropsychological Substrates of Expertise in 
Later Adulthood 


Evidence from neuropsychological studies indicates that experts acquire 
specific mechanisms to adapt maximally to the constraints of their domains. 
Several studies with young experts found specific adaptations such as 


structural changes in cab-drivers’ hippocampal areas (Maguire et al., 2000) or 


string players’ increased representation of the left-hand fingers in the 


somatosensory cortex (Elbert, Pantev, Wienbruch, Rockstroh, & Taub, 1995). 


In a later, longitudinal study Woollett and Maguire (2011) showed that the 
specific adaptations in cab-drivers occurred during the skill acquisition 
process and could thus not be attributed to selection effects. 
Domain-specificity of brain regions supporting acquired expertise was 
also demonstrated in studies with perfumers (Delon-Martin, Plailly, Fonlupt, 


Veyrac, & Royet, 2013) and mental calculators using the abacus (Hanakawa, 


Honda, Okada, Fukuyama, & Shibasaki, 2003). Functional imaging data 


obtained from ten young and middle-aged top performers at the World 


Memory Championships (Maguire, Valentine, Wilding, & Kapur, 2003) 


demonstrated that memory expertise was not related to general intellectual 
abilities or structural differences in the brain, but to intensive usage of spatial 
learning strategies, which was reflected in increased medial parietal and 
retrosplenial cortices and posterior hippocampus activation. Memory experts 
and controls used the same brain regions during memorizing; however, 
experts presumably made better use of them in the context of spatial memory 


strategies (mostly the Method-of-Loci). In their studies of chess experts 


Bilali¢ and colleagues (Bilali¢, Langner, Erb, & Grodd, 2010) found that the 
left temporal and parietal lateral brain areas activated during chess-specific 
object recognition were typical of general object recognition in novices; 
however, only experts during chess tasks showed a specific pattern of 
bilateral activation of homologous regions (Bilali¢, Kiesel, Pohl, Erb, & 
Grodd, 2011). 

Meister et al. (2005) had young pianists and non-musicians perform 
simple and complex finger sequences. They found that non-musicians 
activated pre-supplementary motor (pre-SMA) and rostral premotor regions 
to control more complex sequences, whereas pianists showed no such extra 
activation to accommodate task complexity. Similar to the results obtained by 
Meister and colleagues, Krampe and colleagues (Krampe, Wenderoth, 
Lavrysen, & Swinnen, forthcoming) found that young and older novices 
recruited prefrontal regions typically associated with domain-general 
functions, when performing complex rhythmic sequencing tasks. In contrast, 
young and older expert musicians alike showed higher activation in the dorsal 
premotor cortex pointing to task-specific representations, which presumably 
protected older experts’ timing skills from age-related decline. In tasks 
measuring domain-general functions, like digit-symbol substitution, digit- 
span, and the Go-—NoGo task, older experts showed normal age-related 
decline. These results are clearly in line with a differential preservation 
account. 

Several studies found structural brain differences in young and middle- 
aged musicians or musically trained individuals compared with controls 
(Gartner et al., 2013; Gaser & Schlaug, 2003). Naturally, cross-sectional 
evidence cannot rule out that brain adaptations existed prior to the acquisition 


of expertise. However, because the observed adaptations were highly specific 


for the domains under investigation the authors of the studies attributed them 
to long-term training processes. Two studies reported direct correlations 
between years of practicing and neural adaptation. Bengtsson et al. (2005) 
found systematic correlations between hours practiced on the piano in 
childhood or adolescence and adult white matter integrity. The size of the 
cortical representation of fingers in the aforementioned study by Elbert et al. 
(1995) was not only specific for the (left) hand predominantly used by string 
players for touch control; it also correlated with musicians’ numbers of years 
since the beginning of formal instrumental training. 

Recent studies reported structural differences between older musically 
trained individuals and novices and related them to domain-general 
processes, like executive control (see later section). Parbery-Clarke et al. 
(2012) measured subcortical (brainstem) response delays to auditory 
stimulation (consonant—vowel speech sounds) in young and older musically 
trained adults and non-musicians. They found the typical age-related 
response delay in non-musicians; however, except for a very brief period 
after stimulus onset, this delay was basically absent in musically trained 
middle-aged and older adults. The authors argue that long-term musical 
training protected the auditory system from age-related decline or at least 
offsets its occurrence. Zendel and Alain (2014) also investigated auditory 
processing in young and older musically trained participants using ERP. They 
found that musically trained participants had an advantage in signal- 
processing of exogenous (peripheral) sources, but both groups showed 
similar age-related declines. When analyzing endogenous processing related 
to auditory attention, they found enhanced activity in older musicians. The 
authors argued that older musicians compensate for peripheral decline in 


primary auditory cortices by increasing attention. In their study of ERP 


patterns during tactile perception tasks Reuter and colleagues (Reuter, 
Voelcker-Rehage, Vieluf, Winneke, & Godde, 2014) found reduced age- 
related changes between early middle-aged and late middle-aged 
professionals with occupations requiring extensive dextral manipulations 


(e.g. dentists or watchmakers) compared with controls. 


Deliberate Practice and Expertise 
Maintenance in Later Adulthood 


Most studies of older experts remain silent as to the costs involved in 
maintaining expertise into adulthood. The “maintenance through deliberate 
practice” account emphasizes the need for continued efforts. However, it is 
subject to at least one alternative explanation. It is feasible that experts 
acquire the critical skills in their domain at younger ages and that related 
mechanisms remain available throughout later adulthood. Thus, deliberate 
practice may well be the key factor in the acquisition phase, whereas 
comparatively little individual effort and investment is necessary to maintain 
high levels of performance thereafter. 

The role of maintenance practice is undisputed in sports. Longitudinal 
data from expert runners show only slight declines in speed between the mid- 
30s and the 60s for those individuals who continue to practice (Tanaka & 


Seals, 2003). Performance falls off at a faster pace at older ages; however, 


this effect is even further delayed in active swimmers, presumably because of 
their smaller risk of injury and lower weight-bearing stress. Pimentel and 


colleagues (Pimentel, Gentile, Tanaka, Seals, & Gates, 2003) found a clear 


advantage in aerobic capacity for endurance-trained men compared with 
sedentary men; rate of decline after age 50 in athletes was directly related to 
practice intensity (as were running times). The power of training to mitigate 
age effects was also evident from a study on amateur marathon or half- 


marathon runners conducted by Leyk and colleagues (Leyk et al., 2010). 


They found that through regular training 25 percent of their 65- to 69-year- 
old runners ran faster than the median of the 20- to 54-year-old group. 

In their study on maintenance through deliberate practice, Krampe and 
Ericsson (1996) studied expert and amateur pianists of different ages, with a 
combination of experimental and psychometric measures of ability, along 
with self-report and diary data recording time investment in deliberate 
practice and other activities. The expertise-related abilities tested comprised 
virtuoso skills like maximum repetitive tapping and speeded multi-finger 
sequencing tasks, but also non-speeded tasks such as memorization of 
sequences and (rated) expressive musical interpretation. In line with results 
for typists and chess experts, the authors found that older professional 
pianists showed normal age-related declines in measures of general 
processing speed such as choice reaction time and speed of digit-symbol 
substitution. However, while age effects within the amateur group with 
regard to expertise-related measures of multiple-finger coordination speed 
were similar to those pertaining to the general speed measures (e.g. choice 
reaction time) they were reduced or fully absent in the expert sample. 
Consistent with the selective maintenance through deliberate practice view, 
the authors showed that maintained performance in old age for speeded 
expertise tasks depended on the amounts of deliberate practice invested in the 
fifth and sixth decade. In novices, but not in experts, measures of general 
processing speed were correlated with performance in the amateur group, 
suggesting that the basis of expertise is decoupled from general abilities, 
particularly if later adulthood is considered. 


In a similar vein Charness and colleagues (Chamess et al., 1996; 


Charness, Tuffiash, Krampe, Reingold, & Vasyukova, 2005) found that chess 


ratings (based on chess tournament performance) in a large sample of rated 


players, covering ages from 20 to 80 years, depended on amounts of 
deliberate practice (more accurately, purposeful practice, given that coaching 
did not play a strong role in that study) far more than on chronological age 
(standardized regression coefficients were —0.38 for age and 0.62 for 
deliberate practice). The effects of purposeful practice were even more 
pronounced in the older players. A regression analysis showed an interaction 
of age and current purposeful practice in predicting current skill level. It 
appeared that older players needed greater current practice than younger 
players to reach equivalent skill levels, again pointing to the need for 
continued investment in maintenance of skills at advanced ages. However, a 
second interaction suggested a trend toward diminishing returns from 
purposeful practice later in life. Increasing amounts of cumulative purposeful 
practice did not reap the same gains in skill level for older players. One 
potential factor might be general age-related slowing in acquiring new 
domain-related information (e.g. Charness, Kelley, Bosman, & Mottram, 
2001). More recently, Vaci, Gula, and Bilali¢ (2015) found that the age when 
chess players stabilized their performance levels after having passed their 
peaks was directly related to tournament activities. In the earlier study by 
Charness et al. (1996) chess players had identified tournament competition as 


an activity similar to purposeful practice in terms of advancing skills. 


Expert Mechanisms as Compensatory Means for Age-Related 
Decline 
One of the most fascinating theoretical perspectives on outstanding 
performance in older age is the idea that older experts compensate for age- 
related declines in certain capacities through the development of specific, 
higher-level mechanisms. This idea motivated the “molar 


equivalence—molecular decomposition approach” (Chamess, 1981a; Over & 


Thomas, 1995; Salthouse, 1984; Westerman, Davies, Glendon, & Stammers, 


1998). This approach entails the study of samples of people in which 
correlations between age and levels of performance (e.g. typing speed, rated 
chess performance) are essentially zero: molar equivalence. Based on the 
decomposition of a complex skill into component processes — molecular 
decomposition — investigators then use differential patterns of age-related 
changes among sub-processes to establish evidence for compensatory 
mechanisms. 

Early evidence indicating compensatory mechanisms came from a study 


on age and chess expertise (Charness, 1981a, 1981b), finding that the quality 


of the chess moves subjects selected for an unfamiliar chess position was 
unrelated to age and closely linked to skill level (Elo chess rating scale). 
Detailed analysis of think-aloud protocols revealed that older experts engaged 
in less extensive search in a move selection task than their younger 
counterparts did, but they selected moves of comparable quality. One 
possible interpretation of these findings is that older players compensate for 
age-related declines in search and retrieval speed with more refined 


knowledge-based processes related to move selection. 


The molar equivalence—molecular decomposition approach was also 
applied by Salthouse (Salthouse, 1984, 1991) in his study with typists. He 
found that across age groups basic components of movement proficiency like 
the rate of repetitively typing the same letter showed a moderate correlation 
to overall typing speed, accounting for 42 percent of the variance. In contrast, 
measures reflecting complex expertise-related mechanisms like the speed of 
typing letters with alternate hands or the eye—hand span (i.e. the number of 
letters they looked ahead prior to executing the actual keystrokes) accounted 
for more than 70 percent of the inter-individual differences in overall typing 
speed. Note that repetitive tapping rate, like other measures of general 
processing speed, showed typical age-related decline in this sample, while the 
correlation between age and overall typing speed was essentially zero. 
Interestingly, older expert typists showed larger eye—hand spans compared 
with their younger counterparts. Salthouse argued that the successful 
maintenance of typing skills in his older expert typists relied on extensive 
anticipation as illustrated by older skilled typists’ longer eye—hand spans. 
Bosman (1993) also provided evidence that older typists compensate for their 
age-related declines in basic motor speed through their extended eye—hand 
spans. 

A related interpretation assumes that aged experts rely differentially on 
component processes that can be more easily maintained at advanced ages. 
The latter view is closer to the selective skill maintenance interpretation 
advanced by Krampe and Ericsson (1996), who argued that older expert 
pianists maintain their levels of performance by selectively training existing 
skills. In either case deliberate practice is seen as necessary to detect 
weaknesses and to develop existing or new skills. Cross-sectional analyses of 


expert mechanisms face the challenge to determine whether older individuals 


deliberately adopted compensatory mechanisms in response to aging, or 
whether mechanisms supporting performance at younger ages were better 
preserved due to a slower age-related decline or due to deliberate practice 
activities. Longitudinal studies are needed to address this challenge. 

More recently, the concept of compensation has also become a central 


topic in the neuropsychology of aging (Hanggi et al., 2015). Several authors 


have suggested that the older adults’ recruitment of symmetrically 
contralateral brain regions when performing the same task as young adults 
constitutes compensatory recruitment (Reuter-Lorenz & Cappell, 2008), 
which distinguishes high-performing adults from those showing pronounced 
decline (Cabeza, Anderson, Locantore, & McIntosh, 2002). Hypothesized 
neural scaffolding processes (Park & Reuter-Lorenz, 2009) potentially 
explain compensatory processes. The compensatory benefit of neural 
scaffolding may also depend on the specific brain regions activated: while 
bilateral recruitment in older adults typically involves (less functional) 
prefrontal areas, young adults and experts are more likely to show bilateral 
activation in posterior regions to accommodate task complexity (Bilali¢ et al., 
2011). 


Level of Skill, Experience, and Domain-Specificity of Maintained 
Skills 

The reported benefits of sustained maintenance practice for older experts and 
the possibility of developing compensatory strategies appear to be good news 
for the successful mastery of everyday life and professional competence in 
the elderly. However, not all studies find a mitigation of age-related decline 
by high levels of skill and several attempts to link expert performance to 
previous engagement in skill-related activities were unsuccessful. Physical 
and mental resources needed for task performance may show differential age- 
related changes as a function of genetic, environmental, and joint (gene x 
environment) influences. Even purposeful practice may not be able to fully 
protect a musician’s acquired dexterity from the ravages of age-related onset 
of arthritis or focal dystonia. 

Studies of age-related changes in leisure activities tend to support the 
claim that age-related losses in basic cognitive abilities result in deficiencies 
in more specific skills. For example, in their study of memory for baseball 
game descriptions Hambrick and Engle (2002) found similar age-related 
differences among individuals with high vs. low levels of domain-relevant 
knowledge (that is, no mitigation). The authors identified inter-individual 
differences and age-related declines in working memory as the critical 


factors. Krampe and colleagues (Krampe, Engbert, & Kliegl, 2001) found 


similar performance problems in older amateur musicians’ rhythmic timing 
as in novices. In her study of perception and memorization of musical 
materials Meinz (2000) found attenuations of negative age effects only in 


some tasks and only in the most experienced participants. When tested for 


basic sensorimotor functions like maintaining a stable stance under sway- 
referenced conditions, martial arts experts holding a black belt outperformed 
sedentary controls and even individuals from disciplines without explicit 
balance components, like runners (Krampe, Smolders, & Doumas, 2014). 
Negative age effects were also smaller in the martial arts group than in the 
sedentary group. At the same time, martial arts experts in their late 60s and 
early 70s performed at weaker levels than sedentary young adults, suggesting 
that high levels of skill in complex movement control do not necessarily 


protect basic sensorimotor functions. 


Salthouse and colleagues (Salthouse, 1991; Salthouse et al., 1990) found 
robust age-related decline in spatial ability, even in older architects who were 
still practicing (see also Lindenberger et al., 1992). Salthouse et al. (1990) 
observed that the occupational relevance that participants attributed to the 
experimental tasks in their study correlated negatively with age, suggesting 
that different types of skilled processes were required for young and older 
architects. In line with the latter assumption, expert pianists’ diary data 
(Krampe & Ericsson, 1996) showed a greater amount of professional 
activities in older compared with young experts, but also a pronounced shift 
in focus (e.g. less practice and more teaching). 

Differential age effects for skill components within a domain were also 
observed in piloting or air traffic control. These domains maximally tax 
experts’ skills to act upon unpredictable events or their task-sharing abilities. 
Morrow et al. (2003) observed poorer air traffic control message recall 
performance of older pilots compared to younger pilots. However, when the 
task was changed to a realistic one that permitted note-taking, that age 
difference disappeared. Similarly, simulator-based flying accuracy in 


response to air traffic control instructions showed age deficits that were 


mediated by age-related differences in working memory measures and speed 
of processing (Taylor et al., 2007). Positive expertise effects on flying 
performance were also observed and attributed to purposeful practice 
differences. However, expertise did not interact with age to reduce age- 
related differences. 

In sum, constraints exist for the maintenance of skills in professional 
and pastime contexts. Older amateur pianists in the Krampe and Ericsson 
(1996) study had up to 40 years of “experience” in playing the piano. In 
contrast, the amount of deliberate practice accumulated by this group was 
less than half of that estimated for young experts, who were 35 years younger 
on average. It is questionable that amateurs have the knowledge and the 
motivation to systematically engage in similar deliberate practice activities as 
experts do. Further, even professional experience does not guarantee that the 
relevant capabilities remain intact in older age. Rather, the available evidence 
suggests that maintaining skills is as effortful as acquiring them in the first 
place, and benefits become increasingly more specific, limited to those skills 
which are actively practiced. 

A second constraint on successful maintenance arises from differential 
sensitivities of different skill components to age-related declines. The above 
studies indicated that the relevant skills as well as the type of activities 
experts engage in vary with age. From the available evidence it is impossible 
to determine whether older experts shifted to certain activities because related 
skills were easier to maintain or whether their professional contexts 
demanded it. Finally, there is also evidence suggesting that the capacity to 
engage in skill-sustaining deliberate practice activities as well as the benefits 
individuals can expect from practice might both be constrained by advancing 


age. 


Age-Related Constraints on Improvement 
through Practice 


It is plausible that expertise but also age should lead to growing knowledge 
about optimal and efficient practice methods. Somewhat different from this 
positive perception, laboratory training research suggests that cognitive 
plasticity, learning rates, and the ultimate performance outcomes decrease in 
later adulthood (Kliegl, Smith, & Baltes, 1989), particularly after the age of 
70 (Singer, Lindenberger, & Baltes, 2003; Yang, Krampe, & Baltes, 2006). 


Consistent with laboratory research, in part because the studies reviewed 


contained many instances of lab research, meta-analyses of job-related 
training as a function of age show moderate to strong negative correlations 
(Callahan, Kiker, & Cross, 2003; Ng & Feldman, 2008). That is, older adults 


seem to benefit less from training than younger ones (Ng & Feldman, 2008), 


or they require specific types of training (self-paced training) to approach the 


degree of benefit found for young adults (Callahan et al., 2003). Such studies 


may not generalize well to acquiring and maintaining expertise because 
training methods used were not individualized for the most part, and likely 
were not aimed at promoting high-level performance. Nonetheless, they may 
speak to increasing difficulty expected for older experts who have to learn 
new techniques that are possibly unrelated to past ones. As mentioned above, 
Charness et al. (1996) also found a weak interaction between age and 
purposeful practice, suggesting a diminishing return for cumulative 


purposeful practice for older chess players. 


In longitudinal research on pilots Kennedy et al. (2015) showed that 
simulator training, particularly intervals between training sessions, affect 
gains for a variety of tasks such as communications, approach, traffic 
avoidance, and responding to emergencies. Gains did not interact with 
expertise level in middle-aged and older pilots but there was less gain for 
older pilots, and gains were influenced by cognitive abilities such as speed 
and executive function. These findings point to potential limitations of 
individual adaptation in less predictable settings or to increasing difficulties 
to counter age-related changes through training. Individuals who monitor 
such changes may in turn show changes in attitudes and motivation toward 
career development and critically, to practice (e.g. Gembris & Heye, 2014). 

The discrepancy between stronger age-related decline in lab tasks 
compared to job tasks may be attributable to differences between usual and 
maximal performance. When we examine maximal performance in real 
settings the most usual pattern observed is a backward inverted J-shaped 
function (Simonton, 2012). For instance, in a highly competitive 
environment, chess playing, within a very elite sample of grand masters, Elo 
(1965, 1986) observed that there was a rapid rise in ability during the teenage 
and young adult years and then about a one standard deviation decline in 
tournament performance from the peak years of the mid-30s to age 65. 
Although a later study using a broader skill range suggested a later peak 
(Roring & Charness, 2007), a recent study has shown a more complicated 
pattern with older expert chess players showing a second inflection point in 
old age with experts showing slower decline than less expert players after a 
period in middle age of faster decline (Vaci et al., 2015). Slower decline rates 
were influenced by greater engagement in chess, in the form of tournament 


participation. 


Another activity in which outstanding performance can be fruitfully 
examined is sports, where the clash between opponents provides strong 
feedback about superiority and inferiority and where the financial incentives 
to excel are extremely high. In sports domains, it is rare to play at top form 
beyond the fourth decade (Schulz & Curnow, 1988). Here too we do not 
usually have easy access to training regimens for participants, so it is difficult 
to know to what extent reduced motivation to maintain intense training or 
biological/physiological factors are responsible for decline. One potential 
explanation is that the time it takes individuals to recuperate from challenging 
practice activities increases with age, and thus limits the total amounts of 
time that can be invested into the maintenance of expertise. While this is most 
obvious in sports, and takes its extreme form in injuries, the argument applies 
to all fields of expertise although it has not received similar attention. In the 
context of the deliberate practice model (Ericsson et al., 1993) this implies a 
substantial change of resource constraints with advancing age. The increasing 
impact of bodily functions and health condition on learning ability and 
cognitive functioning in older age is also evident from research showing the 
benefits of physical fitness for executive functions (Kramer & Willis, 2002) 
and dual-task research showing increasing attentional costs in older adults 
exerted by simple bodily functions like postural control (Boisgontier et al., 
2013). 


Does Expertise Provide General Benefits at 
Advanced Ages? 


There is by now little disagreement in the literature that acquired, domain- 
specific mechanisms support expert performance at any age. From this 
perspective, we would expect little transfer to general intellectual abilities. 
There are, however, cross-sectional results that hint at positive domain- 
general cognitive outcomes of training or practicing in a specific domain. 
Bailey and Sims (2014) studied women between 18 and 77 years of age, who 
were crafters (knitting, sewing, crocheting) at two levels of (self-reported) 
skill. They found different patterns of performance on a spatial abilities test 
(paper folding) as a function of age and expertise. Spatial ability declined 
with age as expected, but an age x expertise interaction indicated that for 
young adults, expertise was unrelated to spatial ability whereas in middle and 
older age groups spatial ability was higher in more skilled crafters. However, 


another study (Cavallini, Comoldi, & Vecchi, 2009) using a variety of 


cognitive tasks (visuospatial, working memory) found no evidence of an 
interaction between skill and age for architects and literary people versus 
controls without such professional expertise. There were simply main effects 
of age (negative) and expertise (positive) on abilities. 

Parbery-Clark et al. (2012) found that older musicians’ advantages in 
auditory processing also extended to the perception of speech in noisy 
contexts. Studies with young adults observed that individuals playing a 
musical instrument even at amateur levels only showed advantages in 


performance monitoring and other executive functions (Bialystok & DePape, 


2009; Jentzsch, Mkrtchian, & Kansal, 2014). Amer and colleagues (Amer, 


Kalender, Hasher, Trehub, & Wong, 2013) compared professional musicians 


aged 50 to 77 years with age- and education-matched controls on near 
transfer (auditory pitch and word Stroop) and far transfer (cognitive control) 
tasks. They found an advantage for musicians in near-transfer tasks for pitch, 
but not for words reflecting the specificity of expert auditory processing. 
However, professional musicians also outperformed controls on several far 
transfer tasks, notably visuospatial span, reading with distraction, and the 
Simon task (but not the Go—NoGo task). The authors interpreted their 
findings as evidence that sustained musical training leads to improved 
cognitive functioning, particularly for cognitive control, because related 
aspects are exercised in many musical contexts. 

Several neuropsychological studies also reported structural differences 
between musicians’ and novices’ brains in young and middle-aged adults, 
which could favor visuospatial working memory and cognitive control (Gaser 
& Schlaug, 2003; Sluming, Brooks, Howard, Downes, & Roberts, 2007) and 


were assumed to reflect far transfer by the authors. The issue of whether there 


are training-induced brain changes in musicians that can transfer to non- 
musical cognitive abilities or compensate for age-related cognitive declines 
was also investigated by Hanna-Pladdy and colleagues (Hanna-Pladdy & 
Gajewski, 2012). These authors found that individuals with at least ten years 
of musical experience had better working memory, naming, visuospatial 
functions, and executive processes in advanced age relative to non-musicians. 
Additional regression analyses suggested that past and recent musical 
activity, but in some cases also education were related to level of 
performance. 


One difficulty in interpreting such cross-sectional research is that prior 


ability profiles, such as superior general working memory capacity, may 
influence who initially participates in and persists with such mentally 
demanding activities. However, some longitudinal studies also point in the 
direction of general gains in cognition for intellectually stimulating work 
environments or leisure activities (e.g. Schooler & Mulatu, 2001). A more 
direct test of the expertise-driven general ability account is provided by 
training studies with randomly assigned participants and repeated tests of 
general abilities. Such studies have indeed been conducted with children, 
reporting small, but reliable benefits of musical training to broader functions 
like IQ or school performance (Moreno et al., 2011; Schellenberg, 2004). 
One study (Seidler et al., 2015) also investigated transfer from domain- 
specific, sensorimotor training in older adults (60-85 years) to cognitive 
abilities associated with executive functions. Older participants in this study, 
who had been randomly assigned to six months of individualized piano 
lessons instead of a non-training control group showed significant 
improvements in Trail-Making B (a measure of executive functions) and 
Digit-Symbol Substitution tests (a measure of processing speed) compared 
with controls. 

By implication, more practice and longer training should lead to greater 
benefits and accumulated effects should be particularly evident in 
professional musicians. The available evidence is at odds with this 
assumption. Typically, cross-sectional IQ differences between musically 
trained and untrained participants were weaker in adults than in children 


(Bialystok & DePape, 2009; Schellenberg, 2011) and absent or inconsistent 


when professional musicians were compared with amateurs (Krampe & 
Ericsson, 1996) or individuals with smaller amounts of musical experience 


(Hanna-Pladdy & Gajewski, 2012). From his own review Schellenberg 


concluded: “In short, cognitive advantages are evident for those who take 
music lessons in addition to everything else, but not for those who study 
music instead of something else” (2011, p. 286). 

In the aging context it is next to impossible to distinguish whether group 
differences in general abilities reflect expertise-driven mitigation or preserved 
differences which existed prior to skill acquisition. A conservative 
interpretation of the available evidence is that long-term investment into 
expertise portrays beneficial effects at a more general level rather than 
supporting direct transfer at the level of cognitive mechanisms. For example, 
the necessity to sustain intense practice regimes and coordinate them with 
other professional demands might well motivate high accomplishments in 
one domain and also tend to provide individuals with resources (e.g. salaries) 
to optimize other life domains (health, recuperation). Along these lines 
Krampe and Baltes (2003) proposed that reciprocal effects emerge at the 
level of metacognition or life-management, in ways such as learning-to-learn, 
optimal time-budgeting of daily activities, or the personal belief that pursuing 


long-term goals pays off eventually. 


Summary and Conclusions 


The evidence reviewed in this chapter illustrates that general cognitive 
abilities, as measured through psychometric tests, are weak correlates of 
expert performance in older age. There is also accumulating evidence that 
accomplishments in later adulthood do not merely reflect the success of 
initial learning. More likely, older experts must actively maintain specific 
skills through deliberate practice efforts. Such maintenance efforts do not 
transfer to the more general cognitive abilities typically assessed in IQ tests. 
The potential for maintenance through deliberate practice is not limited to 
purely knowledge-based performance, but rather extends to skills involving 
speed and accuracy. 

For some time, researchers have started to express concerns about the 
decontextualization of tests designed in the IQ-based tradition and standard 
laboratory tasks as valid indicators of competencies and cognition in the 
elderly (Dixon & Baltes, 1986; Sternberg & Wagner, 1986). Subsequently, 


research strategies changed from searching for correlates of age-related 


decline to identifying mechanisms that support successful aging in expert 
individuals who maintain competencies at high levels. As a result, more 
ecologically based approaches focusing on everyday competencies and real- 
life expertise have emerged, which also attempt to incorporate expert 
performance in a revised concept of intelligence (Horn & Masunaga, 2000; 
Krampe & Baltes, 2003; Sternberg, 1999). 


At the level of societies and culture, declining birth rates and continued 


increases in life expectancy in industrialized countries have prompted the 


rediscovery of older adults in their 60s and even 70s as valuable participants 
in the workforce. Some industries are investing in knowledge preservation 
projects to try to maintain institutional expertise when their aging experts 
retire (Hoffman, Shadbolt, Burton, & Klein, 1995), Considerable efforts are 
now being made by industry and applied researchers to design interventions 
suitable for developing older adults’ potentials and supporting opportunities 
for their lifelong learning (Charness et al., 2001). The good news emerging 
from research on expertise and learning is that older adults can maintain high 
levels of skill through their own purposeful efforts, at least up to the third age 
(i.e. until age 70). To this end, societies’ protective mechanisms, which 
typically guarantee that older employees are more likely to be in stable 
positions than younger ones (Swaen, Kant, van Amelsvoort, & Beurskens, 
2002), provide a context for these individuals to selectively maintain relevant 
skills. However, to the extent that job demands change over time, 
obsolescence of skills becomes a risk unless people continue to engage in 
purposeful practice. There are however, limits imposed on the continued 
investment of resources into skill development that emerge at even more 
advanced ages (the fourth age), which ultimately constrain an individual’s 
success in the long-distance race to achieve and maintain high-level 


expertise. 
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action/feedback loops, 456 


Action-Focused method, 782 


actions 


abstracting within events, 294 


anticipating by others, 677-679 


collaborating with perception, 35 
flexible patterns of, 114 


identification and appraisal of, 293 


regulation of, 114 


without the use of words or speech, 772 


active learners, positioning students as, 114 


activities 


amenable to repeated practice, 398 


central to the domain of entrepreneurship, 397 
defining the essence of a domain, 397 
deliberate, goal-directed, 43 

failing to provide cues for timely feedback, 400 


persistent, mindful engagement in, 393 
for persons with specific levels of expertise, 802 
regularly executing, 397 


as signals, 519 
unintended, undesirable side effects of, 139 
activity and/or intensity levels, start and end times for, 258 


activity diaries, 264 


activity patterns, 236 


activity space, introducing dynamism into, 403 


activity systems, 106 

actors 
deep encoding by professional, 63 
expert, not superior to less skilled individuals for memorizing text, 699 
trained to simulate a disease presentation, 332 


actual competition, experts performing in, 678 


actual performances 
as multifaceted, 774 
of the routine, 444 


acupuncturists, experienced, 341 


adaptability, to new circumstances, 120 


adaptation methodologies, using rapid assessment techniques, 807 
adaptive expertise, 436, 448 


adaptive heuristic decision making, 482 


adaptive learning environments, 806 


adaptive mutations, retained, 796 
adaptive performance, of expert teams, 510 


adaptive performance measurement system, 511 


adaptive support systems, beneficial to expert drivers, 367 


added structures, vs. unstructured formats, 799, 801 


added visuals, vs. symbolic only presentations, 799, 801 


addition, mathematical operations required in, 621 


adrenalin (arousal), 553 

adult(s) 
groups of differing in their knowledge about a given domain, 69 
having a vast expert-like knowledge of themselves, 482 
by their 7th decade of life, 837 


adult aging, 836 
Adult-Decision Making Competence assessment (A-DMC), 486, 488, 489 


components of, 487 


adult experts, systematic differences in the structure of brains, 754 


adult numeracy, modern core collection of essential components of, 488 
adult writing, as knowledge transforming, 423 


adulthood performance, world-class expertise manifesting itself in, 322 


advance information, superior pick up and utilization of, 678 


advanced ages, expertise providing general benefits, 847—849 
advanced learners 


making mistakes in their use of collocations, 646 


providing with minimally guided problem-solving practice, 806 


advanced medical students, remembered more and gave more elaborate 
explanations, 802 


advanced multivariate analysis, 304 


advanced music students, think aloud verbalizations by, 206 


advanced progressive matrix reasoning test, 488 

advanced writers, increase in resting state functional connectivity, 418 
advancement, from concrete to abstract thinking, 422 

“Advancing the Rationality Debate” (Stanovich and West), 486 


adversity 


appraised, 301 


not a predictor of scientific performance, 301 


advertising, discourse of professionalism in, 130 
advisory programs, to assist bankers, 97 
aerial screw 

driven into the air, 819 

Leonardo’s design of, 818, 829-830 


aerobic capacity, clear advantage for endurance-trained men, 842 
aerobic exercise, engaging in intense, 754 

aerobic fitness, 754 

affect 


distinguishing between emotions, 464 
providing a metric for mental simulation, 464 


affective computing, distinguishing individuals’ affective state, 519 


affective intuitions, 496 


affective psychopathology, negatively correlated with development of 
expertise, 224 


affective reactions, to changing flight conditions, 464 


affective responses, motivating the expert, 464 


affective states, increasing for a team, 508 
affective traits, 221, 224 

affixes, added to a root form, 642 
Affordable loss principle, 396 


African master drummers, 542 


age. See also aging 


correlations with levels of performance, 843 


as an entanglement with experience, 357 


expected age of peak performance, 273 
expertise in medicine and, 345 


of first publication, 426 


functions based on career age, 320 


at onset of dystonic symptoms, 565 


at onset of musical activity, 561, 840 


as a poor surrogate for direct experience, 345 


relationships with productivity in work settings, 838 


theories of broad decline, 835 
when chess players stabilized their performance levels, 843 
Age and Achievement (Lehman), 312, 320 


age—creativity relationship, historiometric work on, 320-321 


aged experts, relying on component processes, 844 
age-graded declines, in performance IQ, 837 
agency, 112, 444 
age-performance curves, 320, 32 


-_= 


age-performance studies, concerning world-class expertise, 320 
age-related changes 


in domain-general functions, 837 


in leisure activities, 845 


age-related constraints, 846-847 
age-related decline 


expert mechanisms as compensatory means for, 843-844 


explaining in professional achievement, 745 


in knowledge-rich domains, 839 
in lab tasks compared to job tasks, 847 


age-related differences, among individuals with high vs. low levels of 


domain-relevant knowledge, 845 
age-related loss, of functionality in brain regions, 838 
age-related slowing, 838, 843 
aggregated longitudinal designs, 316 


aggregation levels, analyzing data by including several, 280 

aging. See also adult(s); age; negative age effects; older adults; older experts 
deficits, 17 
expertise and, 835—850 


neuropsychology of, 844 


processes of, 274, 610 


aging brain, general processing speed, intellectual abilities and, 837—838 
agreeableness, minimum level of most strongly related to performance, 515 
Al. See aircraft attitude indicator (AI); artificial intelligence (AI) 

air, Leonardo discovered to be compressible, 829 

Air France 447, loss of control over the Atlantic, 464 


air routes and flight schedules, detailed knowledge of, 721 
air traffic control (ATC) 
adaptation methodologies applied to, 807 


challenging for novice pilots, 729 


experiment with a simplified task, 218 
high-fidelity task, 218 


message recall performance, 845 


pilots learning to communicate with, 723 


air traffic controllers 


developing areas of specialization, 358 


learning to scan radar maps efficiently, 723 
needing to know information related to safely separating aircraft, 719 
talking about “the picture,” 736 


air transport, expertise in, 356 


air vehicles, managing and controlling autonomous, 187 


aircraft accidents in combat, major causes of, 717 
aircraft attitude indicator (AI), 366 


aircraft handling, as a primary manual or psychomotor task, 729 
airline scheduling, 97 

airlines, promoting pilots to captain status, 357 

airplanes, interfering with radio waves, 828 

Aitken, Alexander, 617, 624 


alcohol consumption, facilitated performance on problems requiring creative 
thinking, 816 

alcoholism, “reduced” to an aspect, 133 

Alexander the Great, teacher of, 134 


algebra component, of numeracy, 488 


algebraic manipulation, 154 


algebraic problem-solvers, 154 


algorithms, 174 


All in the Family television show, development of, 302 
all-star games, 506 
AlphaGo program, 93 


alternate form, 215 


alternative hypotheses, 88 

alternative practice schedules, improving geometry performance of secondary 
school students, 807 

ambiguity, 380, 457 


American Express, 97 


American neo-behaviorism, emergence of alternatives to, 63 
American Psychological Association, Publication Manual, 419 
amygdala, 238, 239 


analogical reasoning, 99 


analogous situations, 821 


analogy-making, utility of, 374 


analysis 
of cases, 295 
as key to developing expertise in teaching, 447-448 
of protocols, 201 


required for sense-making, 466 


analysis tasks, for expert systems, 96 
analytic problems, solving, 816 
analytic processing, 235 

analytic skills, 447, 448 

analytic thinking, 817 


analytical and experiential knowledge, integration of, 347 
analytical knowledge, 337 


analytical processes, involved in situation assessment, 457 


“analytical” reasoning, compared to exemplar-based models, 333 


anatomical locations, experts more able to correctly identify, 204 


anchoring and confirmation bias, 724 
anesthesiologists, 180, 715 


angel investors, 399 


angle, embedded in a 3-D cube versus in a 2-D parallelogram, 582 


angry patients, having an effect on accuracy of diagnosis, 347 


animals 
expertise in non-human, 49-55 
many as experts, 54 
peak performance for evolutionarily relevant skills, 53 
socially learning a skill, 51 
using in vitally important occupations, 54 
anomalizing, 456 


antecedents, 87, 88 


anterior cingulate cortex (ACC), 247 


anthropometric characteristics, 276 


anthropometric variables, effects of, 281 


anticipation 


decision making and, 682 


of experts, 362 
judgments, 663 


laboratory studies of using domain-specific stimuli, 678 


of opponents’ actions, 16 

role of in the development of expertise, 688 

skilled pick-up of information tightly coupled to biomechanics constraints, 
680 


studies involving evaluations of visual scenes within MRI scanner, 246 


superior, 677—690 
using standard film-based tests of to classify athletes into high and low 


performing groups, 658 


anticipatory aspects of performance, positive transfer of, 688 


anticipatory behavior, expert advantage in, 677—679 
anticipatory bias, 362 
anticipatory effect, 683 


anticipatory encoding, of patterns, 683 


anticipatory eye movements, produced during interceptive tasks, 685 
anticipatory information, learning incidentally, 689 


anticipatory nature, of elite batsmen, 685 


anticipatory performance, variance explained by, 687 


anticipatory processes, moderating by the observer, 684 
anticipatory skill 
development of, 690 


development or transfer of, 688 
improved by perceptual training, 689 
influence of practice on, 687 
investigating in motor experts, 247 
learning, 688—689 


anticipatory strategies, of expert performers, 684 
anticipatory thinking, 774 


antipersonnel mines, 172 


anxiety 
found in dystonia patients, 565 
influencing importance of different sources of information, 664 


leading to dystonia-triggering behaviors, 564 


pressure increasing about correct steps, 803 


AON (action observation network) 
all three main parts of underpinning experts’ anticipation, 247 
anterior areas activated more during early occlusion in tennis and 


badminton experts, 247 


encompassing premotor and parietal and inferior frontal areas in humans, 
246 


influenced by motor and not visual experience, 246 


apologist experts, 139 


applications, capturing superior performance in games, 749 
Applied Cognitive Task Analysis (ACTA), 457, 458 
Applied Concept Mapping, 458 


appraisals 


of actions in case studies, 294 


of participants’ forecasts, 298 


apprentice and journeyman levels, in between experts and novices, 454 


apprentices 
of craftsmen, 5 
learning tailoring, 117 


learning unaccompanied by any direct guidance, 118 


missing more direct precursors of hazards, 363 


restricted fixation to the scene directly in front of them, 361 
supervised by more experienced professionals, 745 
teaching how to reason, 94 


used artifacts, 118 


apprenticeship 


continuing training through, 425 


in creative writing programs and schools of journalism, 424 


required to create complex artifacts, 46 
in sociological and anthropological work, 21 


apprenticeship model of learning, 24 


appropriacy of use, knowledge of, 643 


approximate work tasks, developing, 114 


aptitude measures, 213, 346 
aptitude tests, 216 
aptitude-treatment interactions, 801 
Archimedes, 134 


architect(s) 


ability to maintain parallel processes of cognition, 374 


different types of skilled processes required for young and older, 845 


experienced approaches, 377 


having a seizure, 375 


sketching to construct physically visible markers, 154 


sought to impose order, 375 


unable as a patient to make the transition from problem structuring to 


problem-solving, 376 
architecture, separating facts from procedures and strategies, 86 
Arc-Segmented Attitude Reference (ASAR), 366 


argumentative essays, of freshmen compared to juniors and seniors, 417 


aristocracy, ancient idea of, 139 
Aristotle, 5, 134 


arithmetic calculation, visual salience and, 153 


“arithmetical association,” of mathematical prodigies, 616 


arithmetical principles, ability to apply selectively, 624 
army infantry officers. See also military officers 
SA and expertise in, 730—733 


art 


instruction manuals, 587 


performance differences in as obvious, 576 


removing from its bourgeois pedestal, 827 

art students 
not self-identifying as strong observational drawers, 589 
student apprentices as, 6 


articles, approaches to, 279 


articulation, of tacit knowledge, 785 
artifacts, 118, 119, 121 


artificial environment, evaluating an individual’s performance, 221 


artificial intelligence (AI) 
debates about the nature of, 26 
dominated by expert systems, 88 
earliest phase of, 86 
expertise in the code, 59-60 


foundational ideas, 84—86 


history of, 86-87 
intelligent assistants, 100 


producing information about the attitude of an aircraft, 367 


programs, 60, 61, 85 


relation to psychology of expertise, 65 
research, 87-89 

artist(s) 
accomplished, 592 


accumulating domain-specific knowledge, 579 


advantage in sensitivity to placing squares appropriately, 588 


archival studies of great, 592 


better at a number of visual tasks, 160 


better at focusing attention on task-relevant information, 587 


compared to non-artists, 579, 592 
displaying expertise, 23 
drawing with their eyes, 578 


engaging perceptual and attentional mechanisms, 591 


enjoying advantages in some aspects of perception, 590 


experienced, perceptual processing of, 582 


experiencing shape constancy, 582, 583 


finding parallel deep changes in the perception of, 591 
harnessing knowledge of the structure of appearances, 584 


identical perceptual processes as non-artists, 579 


outperforming non-artists on perception tasks, 579 


perceptual advantages arising from robust representations of object 
structure in memory, 585 


perceptual advantages viewed as a subset of drawing skills, 580 


producing more motor output per unit of visually encoded material, 591 


rendering the three-dimensional world on a two-dimensional surface, 584 


samples as heterogeneous, 589 
seeing the world differently, 576, 579-580 


sensitive to essential features of an object, 585 


situated at the intersection of a set of complex issues, 592 
solving problems in making depictions, 591 


stressing the importance of perceptual factors in depictive skill, 578 


superior to non-artists in some aspects of visual perception, 579 


systematic eye—hand strategy while segmenting complex lines, 591 
tracings of a face more accurate than those of non-artists, 585 


visually analyzing the world from an early age, 578 


artistic expertise 
differences at very low levels of visual processing, 591 
facilitating creativity, 592 
progress in understanding, 590 
roots of, 577 


study of, touching on a broad spectrum of issues, 592 


understanding motoric aspects of, 591 


artistic interpretation, developing, 539 


artistic knowledge, articulating, 585 


artistic realism, as self-evident, 590 
artistic skill, 587 
artistic work, produced by Max Beckmann and Philip Guston, 302 


artist-scientist, emergence of, 135 


art-making, contemporary modes of not involving observational drawing, 589 
arts and science, peak performance in, 750 


artworks, involving visual degradation, 576 


ascending auditory pathway, at the brainstem level, 552 
Asimov, Isaac, 316 
The Ask, 398 

as the appropriate practice task, 401 


creating spontaneous natural feedback, 399 


as inherently intersubjective, 402 
planned sequence of actions, 404 


for purposeful practice in entrepreneurship, 389, 399, 400 


tracking for small business owners, 403 


Ask cards, turning into a game, 404 
asking, 401, 404 
asks 


in the development of expertise, 400 


inputs necessary to creating a new venture, 398 
types of, 399 


assessment instruments, 340 


assessments 
types of in historiometric inquiries, 314 
using people rank-ordered over multiple measuring occasions, 214 


assimilation and accommodation, of Piaget, 820 


“assistance dilemma,” 797 


assistance dogs, 50 
associated chess positions, recreating critical, 748 
associations 


attaching to retrieval cues, 701 


for substrings, 623 
of a word, 640 


associative hierarchies 


bringing to problem situations, 814 

in Pollock pourings, 827 
associative phase, of skill acquisition, 115 
Assumption-based reasoning, in RAWEFS, 457 


assumptions 


based on the context of the moment, 94 
conveying with strategies, 95 
ATC (air traffic control). See air traffic control (ATC) 
athletes. See also elite athletes; expert athletes; skilled athletes; youth athletes 
development of highly skilled, 9 
differentiating groups of, 310 
displaying expertise, 23 
fitness reduced after weeks of bedrest, 754 


generating actions in a simulator, 72 


having a larger anterior part of the cerebellum, 248 


improving performance to be able to compete at the highest level, 274 


in-depth questionnaires to large numbers of, 261 


monitoring over time, 276 


older, progressing via improvements in cognitive capacities, 273 


performance rated subjectively in team sports, 273 


rating social interaction as enjoyable, 655 


scheduling periods of rest to physical work, 656 
showing video footage of evolving sequences of play, 661 


studying behaviors of when practicing specific skills, 656 


training of pushing beyond comfort zones, 754 


atonal music, imitation by savants, 541 
attained performance 
individual differences in, 751 


relation to accumulated practice, 761—763 
attention 
age-related changes, 838 


applying conscious focalized as limited, 717 


directing during thinking, 816 


directing toward surfaces by drivers, 360 


drawn to the high-precedence operation, 156 


engaging, 256 


focused on only task-relevant ideas, 815 


focusing inward by eliminating distractions, 421 
knowledge directing, 721 


role of in visually guided action, 587 


sharing, as an important underlying skill, 725 


Attention research category, for transportation, 357 


attentional control, assessments of, 493 


attentional narrowing, 719, 730 


attentional strategies, adopting overt, 152 


attributes, of experts specific to a time and place, 771 
attuned responsiveness, 36 


attunement to constraint, ecological concept of, 773 


audiences, performing in front of, 538 


auditors, assessments by expert, 4 

auditory association areas (AA), 554 

auditory attention, enhanced in older musicians, 842 
auditory domain, 551, 559 


auditory expertise, 237 
auditory feedback, 550, 554 
auditory gestalts, 554 


auditory long-term memory, 552 


auditory probe, responding to during writing, 417 


auditory processing 


aspects of, 554 


in musically trained participants using ERP, 842 


neural bases of refined, 557 
auditory skills, 552 
auditory type, of memory, 617 


auditory working memory, 622 


auditory—motor—emotion integration capacity, music-making requiring, 554 
aural representation, of an entire piece, 539 


authentic texts, jumping from graded readers to, 647 


authentication and evaluation area, of sociological and anthropological work, 
pal 
authority, projecting, 21 


authors, age of best work, 751 


autistic savants, musical skills of some, 541 


autobiographies, employed in case studies, 293 


autodidactic learning, 537 


automated application, of expert skills, 803 


automated control systems, 827 


automated performance, 75 
automated sensorimotor skills, 803 


automatic performance, 542 


automaticity 
cognitive, 723 


counteracting tendencies toward, 334 


expert performers counteracting, 752 


freeing up attention resources for SA, 729 
level of “good enough,” 436 


of motor skills in more expert musicians, 561 


premature, 447 


in relation to more cognitive tasks, 723 


as a relevant characteristic of expertise, 722 
role in spoken and written form recognition and production, 642 


of vocabulary knowledge, 639 


automation 
counter-acting tendency to, 74 
model for working with, 100 
negative effect of on SA in driving, 735 


automobile driving, 357 


automobile manufacturing process planning, 97 


autonomous helicopter, interfaces for control of a full-sized, 187 


autonomous phase, of skill acquisition, 115, 116 


autonomy of professionals, 22 
availability bias, 334 
“average maturers,” 276 


average sequential thinking, 515 


“average” teaching, documenting, 434 


aviation 
black hole illusion in, 360 


experienced group making fast decisions, 364 


failures to make accurate projections, 716 


aviation pilots. See pilots 
awareness, 444 


awareness-raising, 368 


Awele (an African “sowing” board game), variants of CHREST applied to, 
600 


axons, branching of, 560 


backfire, building a, 830 
backward chaining, 90, 99 


backward inverted J-shaped function, 847 
Bacon, Roger, on mastering mathematics by self-study, 6 
Bacon, Sir Francis, 6, 87, 135 


“Bacionian era,” 135 


badminton players, brain changes in, 248 
Baduk board game, study on the experts of, 244 


Balinese musicians, learning by ear, 544 


ballet dancers 
activation in the AON when watching ballet, 246 
comparing female and male, 246 


deliberate practice for, 757 


bank managers, tacit knowledge of, 778 


bankers, compared to entrepreneurs, 392 


bar graphs, expert reading of, 153 


bariatric surgery, mortality rate, 346 
Barrington, Lord, 535 

Bartlett, Frederic, 677 

basal ganglia, 243, 556 

base model, 183 

base rates, 220, 335 

baseball 


ardent fans compared to more casual, 69 


high-knowledge individuals exhibiting superior learning for materials, 68 


requirements for hitting, 678 


bases, working out configuration of for DNA, 824 
“basic level,” identifying, 159 
basketball players 

completing a typical pattern recall task, 683 


demonstrating immediate recall of game situations, 705 


differentiating between basketball play patterns, 683 


displaying patterns to expert, recreational and novice, 684 


evaluating the center position, 748 


height and success in, 43 
increased gray matter in parts of the cerebellum and striatum, 248 


inexperienced, learning to improve information pick-up, 686 


perception of body parts, 246 


predicting the fate of free throws, 246 
battalion level of command, correlation between tacit knowledge and 


supervisor ratings, 778 


batting performance, dependent on the skill level of the pitcher, 221 


battlefield commanders, 730 
battles, results for generals, 314 
Bayes’ Theorem, 91 

Bayesian probability theory, subjective expected utility theory and, 479 


Bayesian program, not making calculation errors, 91 
Bayt al-Hikma (“House of Wisdom”), in Baghdad, 136 


beach debris, used as artifacts, 117 


The Beatles, ten-year rule and, 540 


beer-mat knowledge, 25 
“Beethovians,” 418 
beginners. See novice(s) 
behaving system, man viewed as, 64 
behavior(s) 
adapting to a changing environment, 820 


capturing the essence of superior performance, 203 


driven by rewards, 514 


encouraging and promoting certain, 522 
expert, directed toward a target, 151 


judgments on, 142 


sampling, 776 


skilled, involving both action and perception, 155 


behavioral and environmental hazard scenarios, 363 


behavioral decision science, 485 


behavioral genetics, 835 


behavioral understandings, building, 160 


behaviorism, 37 
behaviorist theories, alternatives to, 61 


behaviorists, accounting for rule-governed behavior, 207 


“bDeing-in-the-world,” 34 
beliefs 
of expert musicians, 542 


integrating with values, 486 


The Bell Curve: Intelligence and Class Structure in American Life 
(Herrnstein & Murray), 483 

Berlin Academy of Music, 551 

Berlin Numeracy Components Tests (BNT-C), 488, 489 

Berlin Numeracy Tests, 478 


the “best,” not clear in medicine, 344 


best practices, promulgation of, 448 
best solution, finding in shogi, 243 


best work, likely to appear when the most total work appears, 321 


between-person, or inter-individual relative motion, pick-up of, 661 


bias(es). See also cognitive biases 
never going away, 498 


referring to a tendency not implying error, 477 
in research on teaching, 431 


of traditional methods for measuring expert teams, 518 


bicycles 


turning as a dynamic operation, 828 
Wright brothers’ expertise with, 828 
Bidder, George Parker, 620, 624 


bilateral activation patterns, 248 


bilateral frontal pole, 240 
bilateral neural representation, of cognitive functions in the female brain, 627 


bilateral posterior middle temporal gyrus (PMTG), 241 


bilateral supramarginal gyri (SMG), 241, 243 
Bill and Melinda Gates Foundation, 438 


bimanual coordination, 560 


bimanual motor activity, 550 


bimanual or quadrupedal coordination, 552 


bimodal distribution, Edison’s production following, 296 
Binet, Alfred, 193, 617, 625 

binge writing, 422, 424 

Biocon, 395 

biographical controls, 301 


biographical data 


deleting subjects owing to a lack of necessary, 313 


employed in case studies, 293 


biological differences, between the sexes, 627 
biological insights, emerging, 74 


biological markers, of chess talent, 610 


biological motion information, 661 


biological motion patterns, 689 


biologically primary knowledge, 795 


biological/psychological factors, responsible for decline, 847 


biomechanical differences, between actions with different intentions, 680 


biomedical knowledge, 803 

the Bird-in-the-hand principle, 395 
birds, 828 

birth attendants, skill acquisition of, 119 
“black box,” mind as, 33 


black hole illusion, in aviation, 360 


blackboard model, of reasoning, 99 
“blank slate” perspective, 42 
blind people 


olfactory discrimination of, 238 


relying on touch, 238 
blindfold chess, 193, 605-606. See a 
chess masters playing, 72, 202, 702 
blitz chess, 74 
blocked academics, thoughts of, 422 


Iso chess 


blurry photos, identifying objects depicted in, 580 

board experts, structural changes elusive for, 244 

board games, great burden on the brains of experts, 241 

board patterns, involving groups of pieces, 69 

bodily actions or habits, making thinking possible, 35 

bodily functions, impact on learning ability and cognitive functioning, 847 
bodily space, 35 

body 


under extended strain, 754 


as the vehicle of being in the world, 35 


body parts, representation of distinct in brain regions, 556 


“bogus models,” creating, 183 


“bona fide knower,” refusal to recognize, 22 

“bootstrapping,” 396 

borrowing and reorganizing principle, 796 

bottom-up, data-driven process, attention directed across all relevant 


information, 717 


bottom-up and top-down processes, theoretical and empirical reconciliations, 
986-588 
bottom-up explanations, of drawing skill, 581-583 


bottom-up processing 
consequence of, 238 
described, 580 
versus top-down, 580 
bottom-up view, 584, 587 


boundaries 
defining a group, 21 


in the natural visual environment, 157 


boundary conditions, on the perceptual apparatus, 154 


“boundary objects,” stipulating interdisciplinary work, 133 


bounded rationality, task environment and, 64 


boys 


moving to senior competition, 273 
outperforming girls in mathematical problem-solving, 627 
brain 


accommodating expertise, 235 


activation increasing with degree of difficulty of an imagined motor task, 


998 
activation reflecting organization of numerical functions, 629 


activities shifting calculation strategy, 629 


activity during exposure to odors, 238 


adaptations accompanying musical learning processes, 558 

adapting to the cognitive demands of expertise, 248 

anatomy compared between musicians and age-matched medical students, 
962 


changes associated with acquisition of musical expertise, 550—57 


connections of differing in chess experts, 245 


effects of musical training on, 557—559 


functioning integral to skilled performance, 677 


increase in computational burden, 248 
maladaptive changes due to overtraining, 16 


mechanisms underlying insight, 817 


processes underlying insight in problem-solving, 813 


processing diverse streams of visual information, 576 
processing music thorough the use of imaging techniques, 542 


reacting with structural and functional reorganization, 239 


representing movements, 556 


structural changes associated with expertise, 233-248 
systems for mathematical expertise, 627—629 


brain areas. See also brain regions 


active during encoding in memorizers, 240 


differentiation by radiologists compared to medical students, 237 


involved in a particular kind of expertise, 248 


involved in the different components of face processing, 235 


brain damage, reducing digit span to two, 623 


brain imaging, determination of gray and white matter volume, 559 
brain morphometry study, in professional pianists, 562 
brain networks 

coping with demands of culturally acquired competencies, 628 


for musical sub-skills, 552 
brain plasticity. See also plasticity 
in blind people, 239 


not always beneficial, 563 


occurring on different time scales, 553 


performing music as a driver of, 551—554 


as prerequisite and result of expert performance in musicians, 568-570 


shifting from being beneficial to maladaptive, 567 


studying, 550 
brain regions. See also brain areas 


activating by imagined action, 558 


differential decline in, 838 


enlarged after long-term training, 553 
involved in performing music, 554—557 


subject to modifications from musical practice, 553 


supporting acquired expertise, 841 


supporting writing, 415-416 


brass instruments, without a gender bias, 565 


brass players, larger vital and total lung capacities, 542 
breadth, interrelated with depth, 639 
breakthrough thinking, in an unreasonable world, 817, 818 


breast cancer, diagnosing from reviewing mammograms, 203 


“bridge” building, by the identification of a key concept, 379 
Bridger (Swiss Mountain Dog), 51 


briefing and debriefing alone, not enough to ensure high performing teams, 
509 
brittleness 


of knowledge-based systems, 100 


overcoming, 90 
broad attention, creative thinking and, 816 


Broca’s area, in the frontal lobe, 415 


Brodmann’s areas, respective, 555 


Brunswik, Egon, 777 
Brunswik Symmetry, 223, 225 


Brunswikian tradition, 453 
Brunswik’s Lens Model, 223 

buffer zone, finding an existing, 830 
“burner,” in the light bulb, 818 


bus drivers, not having a larger posterior hippocampus, 245 


bus ticket issuing system, designing an automated, 420 


business consulting problem, requiring social innovation, 297 


business opportunities, 392, 398 
business setting, tacit knowledge and expertise in, 778 
Buxton, Jedediah, 624 


cab-drivers. See also taxi drivers 


structural changes in hippocampal areas of young adults, 841 


Caesarean section, 444 


calculating abilities, requiring a normal numerical “starter kit,” 618 


calculating prodigies, biographical details of, 617 
calculation 
cognitive abilities in non-numerical domains and, 618 


distinction with memory, 620 


of effect sizes, 279, 280 


extraordinary feats in abacus competitions, 626 


quickly executing complicated, 240 


as a separable mental process, 626 
by theatrical calculators compared to cashiers, 625 
without access to a physical abacus, 704 
calculators (human). See also expert calculators; mental calculators 
algorithms minimizing the load on working memory, 622 


attracted experimental psychologists as case studies, 617 
born or made, 629 


developing intimacy with numbers from an early age, 624 
developing techniques for reducing current load, 621 
education of, 625-626 

general memory ability of, 703 


holding many items in mind and also knowing facts about numbers, 620 


incidents awakening the interest of, 624 


motivation and instruction of, 624-625 
calendrical calculation, algorithm for, 625 
Cambridge Handbook of Expertise and Expert Performance, 4, 65 
“Campbell’s law,” 432 
cancer 
probability of, 204 
recurrence after surgery, 346, 748 
Candle problem, 822 


canonical occupational knowledge 


comprising societal requirements, 109 
described, 109 


insufficient alone, 110 


key elements of, 110 


canonical underlying knowledge, in relevant academic disciplines, 107 


capacity, in Galton’s theory of eminence, 629 


capital 
access to, 390 
constraints on, 390 


“Caprices,” for violin by Paganini, 544 


captains in the army, serving as platoon leaders, 730 


carbon burner, 829 


carbon compounds, as possible burners, 818 


card sorting, as a measurement of SMM, 512 


cardiac surgery, with an attending present, 346 


career age 
distinguished from chronological age, 321 


of an individual, 315 


career onset, age at, 520 


career paths, defining in science, 137 


career trajectory, producing an overall, 316 
Carlsen, Magnus, 609 


cars, upright and inverted, 158 


case(s) 
compiling an extensive store of, 460 
defining, 292-293 


drawing inferences about expert performance, 304 


gaining a more concrete and detailed understanding, 304 
interpretation of, 294—295 


of leader performance and also scientific performance, 302 


not prohibiting statistical analysis, 299 


providing an initial diagnosis for, 339 
real-world nature of, 292 


reflecting an event or series of events, 292 


as a Starting point for studies of expertise, 292 
studies of multiple, 299 


viewed as observations of a single individual, 292 


case analyses, 291, 298 


case data 
application of questioned, 292 


exploring contemporary, 305 


case events, 295 
case method, 15, 292—293 


case studies 
accompanied by experimental, psychometric, and field studies, 304 


assessing expert performance among scientists and engineers, 300 


on average individuals, 305 


based on observations or records of exceptional high-level performers, 304 
of creative advances, 823-828 


descriptive in nature, 295 


focused on elements or aspects of expert performance, 298 


garnering information about professionally trained chefs, 305 


historic, describing social innovations, 297 


identifying key capacities underlying expert performance capacities, 303 


interpretation of, 294 


key limitation of, 304 
leading to conclusions which triangulate, 303 


of musicians documenting high achievements, 535 


near versus remote associations reconsidered, 830 


presented in support of the remote-associates view, 828 


regarding theories of leadership, 298 


1 


on SA varying as a function of expertise, 726—7 


of savants, 579 


strength of the inferences permitted by, 304 
in studies of expert performance, 303 


in support of the remote-associates view, 830 


systematic, 302 


understanding expert performance by leaders, 298 


usefulness of, 830 


viewed with suspicion, 303 
case-based reasoning system, 99 
case-study scenarios, 777 
cashiers, better than human calculators, 625 
Catalogus Historarium Particularium, 6 


categorization mapping, 722 


category labels, 160 
catheter insertions, 342 
Cattell, James McKeen, 312 


caudal orbitofrontal cortex, 238 


causal analysis, 297 
causal Ask, 399 


causal chain, 183 


causal explanations, 337 
causal knowledge, 339 


causal mechanisms, decision making skill, 478, 482 


causal networks, students developing, 337 


causality 


complicated by “feedback,” 281 


through prediction rejected by expert entrepreneurs, 400 


causation. See also complex indeterminate causation (CIC) 


in real-world settings, 400 


starting or halting points for complex, 462 
cause—effect hypotheses, explaining deviations with, 447 
cause—effect theories, generation of, 447 
CDM (Critical Decision Method). See Critical Decision Method (CDM) 
cellphones, performance and, 366 
cells of the body, adapting, 754 


central concepts/frameworks, relation between expertise and, 17 
central nervous system 

adaptability of, 553 

exhibiting plastic capacities, 560 

sensitive periods during development of, 551 
central sulcus, 555, 560 


central vision, 686 


centralized leadership, 509 


centralized teams, leaders serving as a hub, 459 
cerebellum, 240, 247, 557 


certain conclusions, 479 


certification, of workers/employees, 132 


certification exercises, performing well during, 173 


certification process, for teachers, 437 
ceteris paribus (if everything else is equal), 217 
Cézanne, Paul, simplifying objects’ forms, 578 


challenging practice activities, recuperation time increasing with age, 847 


challenging situations, experienced workers resolving, 175 


chance, formalizing, 479 
change, assessing, 216 


change idea, planning and implementing, 444 


change over time, 216—220 
Chanute, Octave, 827 


characteristics, not all declining at the same rate, 274 


charismatic leaders, 300 


charismatic leadership style, 299 
Charness, Neil, 698 
Chase, Bill, 697, 698 


Chase and Simon 


assumptions by, 699 


in a classic series of studies, 697 


quantifying superior memory for chess positions, 708 


checkers-playing program, 50, 87 

cheetahs, 53 

chefs, 305 

chemical spectral analysis, AI addressing, 62 

chemical structures, determining from analytic data, 85 
chemistry reference works, Edison used, 829 

chemists, 153, 154 

chess, 234, 701—703. See also blindfold chess 


accumulated amount of time playing, 703 


advantage in pattern encoding, 681 
brief description of, 597 
check relationship, 241 


choosing the best possible move, 598 


as a closed environment, 610 


as computationally complex, 481 


correlation between serious study and chess rating, 760 


decision making and, 758 


deliberate practice in, 75 

developmental issues, 606 

differences in experience, knowledge, and skills, 68 

effects of on academic domains and general cognitive abilities, 607 
expertise in, 597-611, 697 

factors differentiating players of different skill, 257 


higher skill associated with deeper search, 701 
key positions in enabling Masters to acquire a “system of playing 
methods,” 604 


as a model task environment, 597 


novices using reasoning by analogy, 606 


only reliable difference as quality of the move, 438 


pioneering studies of expertise in playing, 200—201 


predictors for current skill level in, 608 

problem-solving minimally affected by concurrent verbal activities, 605 

requiring ability to think about space, movement, time, and to hold a goal 
in mind, 43 


serious solitary study of highly correlated with attained performance, 703 


skilled object and pattern recognition in, 241 


studies of expertise in, 63 


theories making clear-cut empirical predictions, 610 
unique impact on expertise research, 610 
chess boards 


presented with randomly arranged chess pieces, 697 


recalling multiple presented in rapid succession, 70 


chess computer programs, attempting to approximate an optimal solution, 
481 
chess expertise 
achievement of, 43 
factors in the acquisition of, 610 


mental manipulation and simulation of the perceived stimuli, 233 


studies of illustrating cognitive mechanisms, 481 


studying, 62 


chess experts 


brains accommodating highly specialized cognitive processes, 243 


comparing to less skilled chess players, 481 


considering more alternative move sequences, 202 


considering qualitatively different moves, 481 
delaying when evaluating multiple moves, 418 


domain-specific knowledge stored in LTM, 234 


faster at identifying chess pieces, 241 


looking for the best continuations, 234 
memory constrained to regular chess positions, 10 
memory skills, 70 


not requiring cooperation of the chessboard, 431 


superiority largest with meaningful positions, 606 
visual-perceptual processes underlying mastery, 73 


chess games 


experimenter reading sequences of moves from multiple, 203 


merely playing not providing immediate feedback after each move, 703 


not every move equally important, 748 


playing a larger number of on the internet correlation with chess skill, 753 


chess grandmasters 


recognizing tens of thousands of board configurations, 796 
seeing the board differently, 619 


simultaneously playing hundreds of games, 480 


chess masters 
following multiple games, 72 


integrating piecemeal information presented auditorily, 702 


making better moves, 438 


mentally generating chess positions associated with multiple chess games, 
203 

mentally updating multiple chess games, 702 

playing blindfold, 72, 202, 702 


problem-solving expertise not generalizing to other tasks, 89 


recalled more pieces correctly for positions from chess games, 697 


recalling a series of different chess positions, 73 


recalling information from 5—10 chess positions, 698 


recalling positions after a single tachistoscopic presentation, 623 


recognizing 50,000 patterns of chess pieces from master-level games, 820 
seminal study of skills of, 820 


thinking aloud during a tournament match, 200 


chess moves 
all not equally important, 749 
enormous number of possible, 598 
quality of, 844 


chess openings, studying, 758 


chess performance 
affected separately by IQ and practice, 609 


superior captured by presenting chess positions, 699 


chess pieces, possessing characteristic functions, 243 


chess planning skills, not transferring to solving the Tower of London, 607 


chess players. See also abilities, of chess players; expert chess players; older 


chess players; skilled chess players 
ability to play “blindfolded,” 193 


amateurs less able to extend their search beyond the top left quadrant of the 
board, 604 
asking to think aloud, 597 


aspects of cognitive mechanisms and processes, 698 


average score highly correlated with official chess ratings, 749 
better, having a larger visual field, 599 


choosing the best moves to play, 598 


consistently winning tournament games, 67 
descriptions of games centered on key positions, 604 
detecting strategically important configurations of chess pieces, 579 


doing little search yet finding strong moves, 598 


engagement in planning, 702 


evidence for extensive planning and evaluation of consequences by expert, 
72 

examined the chess position and then generated some promising moves, 
701 


fixating more on the edges of squares than weaker players did, 599 


forming new relational patterns for unusual piece placements, 603 


generating moves selectively, 598 
highly skilled, 201 
holding chunks in STM, 697 


identified tournament competition as an activity similar to purposeful 


practice, 845 


increasing ability to select the best move, 699 


longitudinal investigation of a Canadian, 602 
male, 18% not right-handers, 610 


memorized pieces better when functionally relevant, 605 


most proficient not thinking further ahead than less skilled players, 597 


not spending time practicing memory for briefly presented chess positions, 
703 

perceptual skill not helping them memorize visual shapes unrelated to 
chess, 607 

playing speeded chess, 201 


problem-solving behavior specializing in two different chess openings, 602 


recalling locations of pieces, 202 


recognizing patterns of chess pieces (chunks), 697 


remembering multiple boards, 600 
searching selectively among alternative moves, 598 
skill levels determined by the outcomes of 20—40 matches in chess 


tournaments, 748 


study of master level and less accomplished, 68 
superior move selection skills, 837 
tending to use less risky strategies when playing against stronger players in 


rapid-transit games, 606 


thinking aloud while selecting the best moves, 74, 200 


trying to select the best move, 749 


twelve years of age as a tipping point for, 609 


weaker not able to adapt search strategy to the requirements of the task 
environment, 602 
chess positions 
ability to select the best move for presented, 13 


from actual games of chess masters, 200 


best players’ ability to rapidly perceive the relevant structure of the 
presented, 201 


with critical moves, 749 


critical taken from real games, 749 


deeper understanding of the structure of, 193 


encoding and manipulating internal representations of, 203 


generating an accurate memory representation of, 702 


memory for briefly presented markedly differentiated skill levels, 597 


mentally transforming and then planning long sequences of moves, 703 


planning for difficult, 201 

presenting briefly (5 s) and requesting immediate recall of each position, 
697 

presenting to players with the explicit task of finding the best next move, 
701 


randomization of, tapped into pattern recognition processes, 242 


recall of randomly scrambled equally poor regardless of skill, 697 


storage of briefly presented, restricted to STM, 698 
superior ability of skilled chess players to image mentally, 202 


chess problems, presenting with think-aloud protocols, 200 


chess ratings 


in a large sample of rated players, 843 


measuring, 200 


chess representations, 605 
chess skill 


correlated with the quality of chosen move, 603 


positive correlations with fluid reasoning, 607 


relating archival data about international chess skill ratings to group and 


individual practice amounts, 262 


relationship with cognitive ability, 607 
chess-playing computer programs, getting objective scoring of different 


moves from, 749 


chess-playing computers, best move objectively determined by, 200 


chess-specific object recognition, 841 

chess-specific task areas, more activated in experts, 241 
Chicago Manual of Style, 419 

child prodigies 


biographical accounts of, 536 
in chess, 597 
special attention to, 535 
child seat, in the IDEO shopping cart, 824 


childhood and adolescence, development during necessary for attaining 


highest level of achievement, 750 


childhood story writing, common for writers, 426 


children 


born with extensive bilateral cataracts, 151 


distinguishing writing from drawing, 422 
with extensive knowledge of chess and dinosaurs, 12 


intensive musical training bringing about lifelong change, 569 


introduced to the musical domain in an informal phase, 540 
practicing by themselves, 539 


recreating types of activities seen during match play, 659 


subjected to extensive abacus training, 626 
talented, practicing more, 537 
China, competitive piano instruction system, 544 


Chinese chess board game, nucleus caudatus smaller in experts, 244 


Chinese teachers, emphasizing relationship with students, 442 


chi-square goodness-of-fit tests, 279 
choice outcome modeling, 481 


choice path independence/consistency, 486 


choices 
always entailing some risk and uncertainty, 493 
guided by acceptable downsides, 396 
choose-a-move task, using full chessboards, 599 


chordal notes, distributing on the keyboard, 540 


CHREST (Chunk Hierarchy and REtrieval STructures) computer program, 
600, 602 


chronic pain, 567 


chronological age of an expert, 314 


chunking 

rates of, 610 

replicated in other domains, 69 
chunking theory, 600 


chunks 
defined, 69 


experts having more complex, 697 


giving access to information, 600 


recognizing and encoding important features, 579 
reflected a deeper meaningful structure, 69 
CIC. See complex indeterminate causation (CIC) 
cingulate gyrus, 244, 556 
cingulate motor area (CMA), 555, 556 


circular objects, projecting to the retina as ellipses, 581 


“circulation of elites,” 139 


civilian aircraft pilots, 109 


class inclusion illusion task battery, 489 
class of movements, expertise in producing, 681 


classes of situations, instantly recognizing known, 722 


classical music 
composition of, 318 
instrumentalists having to master the most demanding repertoire as 
teenagers, 540 


majority of patients suffering from focal dystonia performing, 566 


classical music composers, 319. See also composers 
expertise acquisition in, 315 


sample of eminent, 319 


classical musicians, solitary practice of, 538 
the “classical” number area, in the brain, 628 


classical repertoire, large portion of practice as solitary, 538 


classification 


issues of, 228 


of prior solutions, 381 
classroom 


categorization of what’s important in, 443 


expertise in, 440—443 


lesson, effectiveness of, 433 
management and organization, 441 


settings, characterized by words, 642 


teaching, narrowing focus to, 432 
Classroom Video Analysis (CVA) assessment, 441 
“Clerical/Conventional” trait complex, 226 
climatic variation, contributing to human brain expansion, 41 


clinical cases, portraying similar patient presentations, 339 


clinical decision making, assessed by script concordance tests, 777 
clinical diagnostic reasoning, 332—336 


approaches to teaching, 336—339 


clinical education, dissociating from the clinical setting, 338 


clinical medicine, creating systematic approaches to learning, 332 
clinical problems, 336, 338 
clinical reasoning 


approaches based on alternative diagnoses for teaching, 339 


curriculum strategies to enhance, 337—338 
developing through education, 336-337 


drawing on a set of canonical factors, 110 


levels of representation acquired to support, 205 


meta-analyses of, 753 
clinical teachers, 785 
CLIPS, 89 


closed loops, in the Flexecution model, 462 


“closed mind,” of a team, 514 
closed sports, 666, 705 

clustering, of genius and talent, 318 
Cmap Tools, 467 


Coach Assessment Instrument, 258 


coaches 
assessing, 258, 755, 763 


collecting information about parents’ height and maturation status, 752 


creating conditions producing the best performance during practice, 668 


encouraging self-directed practice, 668 


individualized practice without, 758 


necessity of having, debated in the chess literature, 606 


objective archival data on, 305 
providing instruction explicitly and frequently, 668 


ratings of elite players, 687 


recruiting individuals for successful teams, 747 
time use of expert, 258 

coaching 
expertise, nuances of, 260 


as a much-needed resource, 446 


in soccer, 668 


Statistics as easily accessible, 305 


theory versus practice divide in, 668 


coach-led individual training, 760 


coach-led play, 658 
coach-led practice, 659, 757 
Co-creative or effectual Ask, 400 


co-evolution 
in computer modeling of design processes, 378 
interleaving problem specification with solution development, 384 


from within the problem space, 378 


co-evolving problem-solution, 378-379 


cognition. See also human cognition; shared cognition; team cognition 
expert, as the “goal state” for education, 64 


focus on explicit and structured aspects of, 166 
important to expertise in driving or flying, 356 


modeling more broadly, 84 
models for individual, 168 


role in creative thinking, 817 


situated in a much larger world of experience, 94 


Cognition Requirements Table, 170, 180 


Cognition Technology and Work, special issues in, 454 


cognitive (intellectual) resources, long-term investment of, 229 


cognitive abilities 
breed differences in, 52 
correlation with performance of beginners, 707 


dog breeds not actually differing in, 54 

high levels of not generally required for skilled or expert decision making, 
476 

intelligence and, 763 


measures as general, broad, or specific, 222 
reanalysis of factor-analytic studies on, 484 
tests of, 489 


underlying expertise in SA, 725—726 
cognitive activities 
following a formal process of reasoning, 165 


involved in predicting hazards, 734 
cognitive adaptations, in music, 541—542 
cognitive advantages, evident for those who take music lessons, 848 


cognitive aging research, 837 


cognitive analyses, 105, 170 
cognitive apprenticeship, 113 


cognitive architecture, 795—797, 802 


cognitive automaticity, 725 
cognitive biases. See also bias(es) 
arising from the use of heuristics, 334 


errors resulting from hard-wired, 336 


experimentally induced, 334 


inherent in human reasoning, 336 
role in reasoning, 335 


teaching to reduce diagnostic errors, 335 


cognitive changes, corresponding to brain changes, 837 


cognitive competencies, systems of, 42 


cognitive conceptualizations, of tacit-knowledge acquisition, 782 


cognitive constructs, 171, 172 


cognitive control (or executive control), components of, 837 


cognitive correlates of expertise, resulting from extended deliberate practice, 
105 
cognitive cost, 382, 383 


cognitive decline, 357, 848 


cognitive demands, of writing, 414—416 


cognitive differences, underlying the SA abilities of experts, 725 
cognitive disinhibition, 815—816, 831 


cognitive domain, 234 


cognitive dynamics, in skilled decision making, 480 
cognitive ecology of expertise, 51 


cognitive elite, 484 


Cognitive Engineering and Decision Making Technical Group, 187 


cognitive engineers, 168 
cognitive expertise 
connecting incoming stimuli with existing knowledge structures in the 
LTM, 239-245 


parietal areas in, 248 


structural brain changes in, 244—245 


cognitive fidelity, 460 


cognitive functions 


overloaded by the demands of composing an extended text, 413 


some domains less affected by aging than others, 838 


within a specific challenging incident, 176 


cognitive impulsivity, assessment of, 489 


cognitive inhibition, being less subject to, 815 
cognitive load 


higher levels of for experts presented with prompts, 799 


managing, 417—419 


measures of, 793 
reduction in initial levels of, 800 
cognitive load theory, as a framework for the expertise reversal effect, 
794—797 


cognitive mechanisms, of expertise, 233-235 


cognitive models 


for individuals and teams, 169 


of specific task performance variations, 60 
cognitive operations, in chess having a time cost, 605 
cognitive or intellectual ability, psychological measurements in, 222 
cognitive performance 


creating systems supporting, 467 


effect of expertise on, 735 
not capturing the exact path of synapses of actual thought, 93 


cognitive perspective, inciting to focus on instruction, 121 


cognitive phase, of skill acquisition, 115 
cognitive plasticity, decreasing in later adulthood, 846 
cognitive processes 


acquiring biologically secondary information, 795 


changes in mediating performance, 806 


comprehensive analyses of the full range of underlying, 494 
described, 178 
differences explaining the superior selection of moves by world-class 


players, 201 


generating thoughts corresponding to the required explanations and 
descriptions, 196 
high-level, 662—663 


inhibiting the performance of experts while enhancing the performance of 


novices, 808 


mediating answers to existing questionnaires, 207 


mediating superior digit-span performance, 199—200 


retrospective reports on during a memory trial, 701 


of tacit-knowledge acquisition, 782 


training those involved in insight and learning from context, 783 


used to transition from one cognitive state to another, 180 


cognitive processing, progression from formal to informal, 166 
cognitive psychology, 60—64 


cognitive researchers, laboratory experiments measuring “microcognitive’ 


functions, 461 


cognitive science 
critical role of knowledge base in cognition and learning, 796 
defined, 61 


mainstream, 171 


“representing” constructs of novice and expert deliberations, 63 
Cognitive Science journal, 61 
cognitive science movement, 63 
Cognitive Science Society, 61 


cognitive shortcuts or heuristics, Type 1 reasoning based on, 333 


“cognitive snap,” 818 
cognitive states, 178, 179, 180 


cognitive strategies, of writers, 418 


cognitive structures, 33 
cognitive structures and skills, acquired with training, 67 
cognitive style and environment, promoting experiential learning, 776 


cognitive support, enhanced, 180 


cognitive support systems, 171 


Cognitive Systems Engineering 


application areas for, 186—187 


drawing on theoretical or conceptual paradigms, 166 
historical background, 165-167 

knowledge elicitation within, 167—170 

models for, 166 


understanding the nature of expertise in work practice, 167 


cognitive task analysis (CTA) 


creating programs for training expertise, 458 


of intelligence work, 172 
methods, 458 


simulation exercises and, 460 


cognitive tools, developed from a formal analysis, 166 


cognitive training, 173 
cognitive traits, 221, 222—224 


cognitive turn, signaled active involvement of the mind, 37 

cognitive units, expertise involving larger and more integrated, 68-69 
cognitive work, performed with high levels of proficiency, 167 
Cognitive Work Analysis, 167, 170 


cognitive work systems, 188 


cognitive/intellectual abilities, 218, 222 


cognitive/intellectual correlations, 223 


cognitive-motor tasks, 835 


cognitive-theoretic language, 168 


cognitivism, perspective of, 106 


coherence, establishing, 416 


coherence standards, 478 


cohesion, of team members, 508 


collaboration 
appraised for scientists, 301 
between specialists, 22 
collective efficacy and team performance, curvilinear relationship between, 
ol) 
collective leadership, theory of, 298 


collective or shared capability, designing as, 372 


collective task engagement in creativity tasks, 520 


collectivistic leadership events, identified, 299 


college admissions tests, measuring wisdom-based skills, 779 


college students, tacit knowledge-acquisition processes and, 782 
Collins, Alan, 61 


collocations, 644 


judged acceptable by native speakers, 646 


occurrence in language, 643 


second language learners tending to use large numbers of, 646 
of a word, 640 
co-located teams, 168, 458 


colon tissue, microscopic images of, 204 


color cueing, 689 


color vision, normal or defective, 618 
colored patch, highlighting locations for directing gaze, 689 


combat readiness status, gathering information on, 732 


commanders, preparing for enemy attacks, 723 


commercial airline pilots, attention to cues signaling deteriorating weather, 
225 


commercial driving privileges, becoming attainable, 357 


“a committee of examiners,” satisfying to become a master, 6 
common arrangements, without explicit agreement or discussion, 772 
common coding view, evidence supporting, 681 


common ground, team members ensuring, 459 


common identity, through occupational and professional socialization, 130 
common practice activities, duration of, 761 


common problems, no difference in accuracy for residents and experts, 205 


common sense understandings, of expertise, 25 
common variance, between predictors, 225 
communal CMC, 517 


communality, among predictors and trait complexes, 225-228 


communication 
as a critical component of teamwork, 511 
network and leader skills, 299, 732 
pivotal role of, 459 


protocols, 459 
styles, 299 
tasks, 785 


team members not exhibiting need for explicit, 511 


communicative purposes, of formulaic language, 645 


communities of practice, 23, 781 


community development, experts fostering, 770 


community of knowers, 22 


company, building for an imaginary product, 405 


comparative advantage, 45 
compensation 
in the neuropsychology of aging, 844 


between performance characteristics, 275 


compensation account, assuming that older experts actively acquire new 
mechanisms, 839 
“compensation phenomenon,” 666 


compensatory behaviors, of drivers, 357 


compensatory benefit, of neural scaffolding, 844 


compensatory mechanisms, establishing evidence for, 844 
compensatory strategies, 357 


competence, 142 


assessment, 486 


improvement, 397 


competencies 
developing evolutionarily novel, secondary, 42 


display of multiple, 314 


competitions, 764 
competitive environment, retaining “currency” in, 666 


competitors, for an imaginary game, 407 


compilational emergence, of a team of experts, 506 
compiled knowledge, 106 
compiled procedures, with minimal reliance on conscious memory, 116 


complete dataset, required to do analyses, 280 


completeness, proof of not possible for cognitive engineers, 168 


completion strategy, 806 
complex calculation, 623 


complex cognitive behaviors, developing to a level of automaticity, 723 


complex computation, 628 


complex environment, finding a way in, 244 
complex hand postures, 552 
complex indeterminate causation (CIC). See also causation 


connection with, 400 


internalizing, 394 
inverting from a liability to an asset, 401 


purposeful practice in domains characterized by, 400-403 


complex information, presenting, 800 


complex interaction, between cognitive processes during anticipation, 663 
complex patterns in memory, as the essential factor for development of 


expertise, 541 


complex perception, cases of, 152 


complex pieces of music, playing of, 555 
complex plans, experts generating during activities, 696 


complex rhythmic sequencing tasks, 841 


complex rhythms, PMA responsible for processing, 556 


complex situations and structures, aligning mental models and physical 
displays, 153 
complex systems, 443, 444 


complex tasks, brain plasticity observed for, 553 
complexity 

Carrying a price, 357 

of the chess problem, 602 


cognitive limit to the amount of, 380 


in design thinking, 376 
component processes, decomposition of a complex skill into, 843 


components, 643 


components or dimensions approach, 640 
composers. See also classical music composers 
faster start for, 319 
“10-year rule,” 540 
think aloud verbalizations by, 206 


composing processes, demands on working memory, 417 
composition, 414, 418, 516 


compositional emergence, of a team of experts, 506 


compositionality, 623 


compound remote associates (CRA) problems, 819, 831 


comprehension 


benefits of imprecise, automatic aspects of, 483 


factors other than coverage contributing to full, 636 


lexical coverage needed for successful, 635 
comprehensive adult numeracy framework, 488 


computation, 61, 625 


computational devices, 59, 61 
computational frame-based knowledge representation, 61 
computational knowledge representation language, 61 


computational models 


accounting for task performance, 197 

focused on capturing methods and forms of exceptional performance, 62 
of problem-solving, 605 

psychology embraced as a method, 60 


regenerating human performance on well-defined tasks, 194 


computational procedures, 96 
computational question-answering system, 61 
computer(s) 


brought renewed interest in human cognition, 194 


modeling the expertise of recognized experts, 99 
not socialized into the relevant community, 26 
processing “symbols and symbol structures,” 59 


providing more efficient training tools, 607 


searching selectively among alternative chess moves, 598 


standing as formal models of human cognition, 61 


computer chess programs, achieving high-level play, 598 


computer game, of entrepreneurship for marketing exercise, 405 


computer mediated communication (CMC), 517 
computer models 
designed to represent artificial methods, 60 
developed by Herbert Simon and Allen Newell, 59 
encoding extracted knowledge in, 76 
in support of various cognitive construct, 64 


computer networking training simulation, 800 


computer programs 
experimenting with variations, 86 
forcing precision, 100 


modeling the human problem-solving processes, 60 


performing intellectually challenging tasks, 696 

simulating aspects of thought processes, 696 
computer science 

close collaboration with cognitive psychology, 61 


expert systems and, 84—86 


computer software, replicating cheaply, 96 
computer tools, 465 


computer-based environments, 419 


computer-based tutors, 807 


computerization, 171, 172 

Computers and Thought (Feigenbaum and Feldman), 60 
computing devices, design of portable and wearable, 467 
conative traits, 222, 224-225 


concentration, 445, 761 


concept and referents, of a word, 640 
concept formation, 61, 63 
Concept Map(s), 176—178 

building, 176 

capturing expert knowledge, 176 


constructed by medical resident groups, 802 


created in social network analysis, 177 
of the field of Social Network Analysis, 176 


as a graphical representation of abstract knowledge, 178 


multiple needed to represent an entire domain, 178 


organizing expert knowledge, 178 


showing gaps in student knowledge, 176 


Concept Mapping, 168 


generating models of abstract knowledge supporting cognition, 176 


including protocols for elicitation, 170 
aS a Measurement of SMM, 512 


representing and communicating complex problems and solutions, 467 


representing practitioner knowledge of concepts and their relations, 182 


concepts. See also abstract concepts; self-concept; semantic concepts; 


solution concepts 
realization of a link between, 814 


underlying cognition, 170 


conceptual design, fixation hindering, 382 


conceptual framework (“ontology”), 92 
conceptual knowledge, role in chess expertise, 604 
conceptual models 


associated with each of the Cognitive Systems Engineering paradigms, 166 


explaining team processes, 508 


(ep) 


conceptualizations, of learning processes, 1 15—11 


conclusions 


certain, 479 


experts explaining after the fact, 94 
concrete words, 417 


concrete working knowledge, 133 


concurrent approaches, 257—260 


concurrent chanting, minimal effects of, 199 


concurrent validity, 263 


concurrent-validation assessment, 216 


concurrent-validity procedures, 216 


condition-action statements, 773 
conditional rules, 87, 88 
Condition-and-Action method, 782 
Condition-Focused method, 782 


conditions, enabling, 71 
“conditions for optimal learning and improvement of performance,” evidence 
of, 755 


conductors. See orchestra conductors 


conference, first using the word “expertise” in its title, 65 
confidence 


of expert teams, 509 


in one’s abilities, 225 


under/over, 487 


confidence calibration, 486, 490 


confidence factor, associated with a rule, 91 


configural operationalization, 515, 516, 521 


conflict 
between democratic control and rational administration through experts, 
i139 


in virtual teams, 518 


conflict management, 518 
confusion, as a critical part of deeper learning, 435 


congenital disability, for arithmetic, 629 


congruence, maximizing, 228 


congruent learning style and work setting, among Malaysian public sector 
employees, 776 
connections 


among unrelated ideas in the unconscious, 814 


of concepts and understanding to the practice of routines, 445 
connective thinking, improving team innovation through cooperative 
learning, 515 


conscious memory, deploying in monitoring and evaluating actions, 116 
conscious perception, of the environment not, 581 
consciousness 


altering to facilitate entry in a flow state, 421 


attribution of to any entity besides adult human language users, 49 


as being-towards-the-thing through the intermediary of the body, 34 
consequence events, 723 
consequences, 7/2 
consequents, 87, 88 
consistency 


accounting of, 214 


of decisions, 486 


as a signal, 519 


conspecifics, 51 
constraint analysis, impact on expert performance, 302 
constraints 
on acquiring high levels of performance, 77 
applying in the situation at hand, 302 


being attuned to, enabling consistent goal accomplishment, 773 


existing for maintenance of skills, 846 


experimental studies of, 302—303 


identification of, 298 
important for the maintenance of expert performance, 837 


imposed by artists on their work, 302 


“introduced” by the designer from domain knowledge, 378 


less restrictive resulting in more viable or original products, 303 
manipulating for solutions of higher quality, originality, and elegance, 303 


promoting attunement to, 781 


constraints of use, of words, 540, 643 


constraints-based perspective, 343 


construct de-confliction, opportunities for, 520-522 


construct validity, as the central validity question, 215 


construction technique, for a knowledge-based system, 85 


content knowledge 
American schools heavily emphasizing the development of, 780 
for teachers, 441 


content presentation, improving, 36 
content “problem space,” 414 
“content specificity” 

finding of, 332 


in the medical domain, 344 


content validity, 215 

context 
effects of, 461, 662 
making invalid assumptions about, 94 
role of, 662 


situation specific and non-situation specific, 665 


context-dependent variability, 360 


contextual expertise, 343 


contextual information, 662 
contextual nature, of teaching, 438 
contingencies 


in expert performance, 302 


shaping expert performance, 291 


contingency planning 
lack of, 733 


in many domains, 724 


continuing interactions, producing a new combination of ideas, 814 
continuous variable, virtuality as, 516 
contributions 


depending on high-level expertise, 26 


outside the individual, 105 


contributory expert, 25 

contributory expertise, 25 

contributory experts, as preferred participants, 26 
control orientation, in effectuation, 395 

control system, developed by the Wright brothers, 827 
Control Task Analysis, 178 


control variables 


inferences and, 295 


permitting statistical adjustment, 316 


controlled attention, increasing to step-by-step skill execution, 803 


controlled processing, versus routine processing, 73—74 


controls, 295 


controversy, periods of, 139 


conventional numeracy, 488, 490 


conventionalization, 644 


convergent validity, 215, 263 
conversation topics, word families for, 637 
cooks, 110 


cooperative primates, humans as, 40 


co-opting, impacting visual perception, 155 
coordination, 508, 512, 518 


Copernicus, Nicolaus, 134 


coping strategies, 457 


co-production of knowledge, 138 


“copy if better” strategy, 51 


core content standards, student learning of, 440 


core decision skill components, 486 


core interest themes, 224 
core-task job performance and creativity, as unrelated to age, 838 


corpus callosum, 244, 561 


correlated cues, 364 


correlational data analyses, historiometrics depending on, 313 


correlational method, lacking the power of causal inference, 322 


correlations, 279 


between age and overall typing speed, 844 


with measures of limited variability attenuating, 220 


pattern of declining, 221 


cortical reorganization, 542 


cortical representation of fingers, 842 


cortical somatosensory representation, of fingers or lips, 567 


cortical thickness, in experts, 245 


corticospinal tract, connecting primary motor areas, 560 


cost estimates, for marketing the imaginary product, 407 


cost savings, major internal, 96 

cost-benefit analysis, needed with any talent identification model, 667 
costly signals, especially important in humans, 44 

counter-elites, emergence of, 139 

counting, 617, 622 


courses of action (COAs) 


not specifying alternate, 733 


recognizing, 386 
typical, 457 


court experts, as witnesses, testifying knowledge, 141 


covariates, time-varying, 281 


coverage figure, 636 


coworkers, support from, 522 

Cox, Catharine, 312 

CRA problems, 819, 831 

crafters, at two levels of (self-reported) skill, 847 


crafting knowledge, for a specific audience, 423 
cramping muscles, 568 


Crazy Quilt principle, 394 


creative achievement, for Edison, 296 


creative activities, deliberate practice and, 75 


creative advances 


building on expertise, 812 
case studies of, 823-828 


developing from attempts to apply expertise to the situation, 832 


evolving out of attempts to apply knowledge to the new situation, 813 


never based directly on remote transfer, 821 


transfer of knowledge underlying large-scale, 831 


creative composition, activations lateralized, 418 
creative design, 378 


creative expertise, domains entailing, 322 


creative experts, treating problems as “harder” problems, 386 


“creative explosion,” 30,000 years ago, 576 
creative ideas, arising from restructuring a problematic situation, 817 


creative imagination, as unstructured, 813 


creative individuals, 815 


creative leap, bridging partial models, 379 
creative output, empirical relation between quantity and quality of, 311 


creative people, 815 


creative problem-solving, 301 


creative processes, close links with co-evolution, 378 
creative production, 820 


creative recombination, as insight, 813 


creative re-interpretation, reaching, 381 


creative response, to a situation, 516 


creative story, writing using unrelated words, 416 


creative thinking 
defined, 813 
deliberate practice and, 831 


depending on the rejection of expertise, 817 


expertise and structured imagination in, 812-832 


expertise in, 820—828 


people solving problems requiring, 813 
remote associates in, 813-820 


stimuli demanding, 814 


types of transfer of expertise in, 820-821 
utilizing remote associations, 814 


creative writers, 418, 421 


creative writing, brain activation, 416 
creativity 
artistic expertise and skill facilitating, 592 


bringing together unrelated ideas, 819 


combining associative elements, 814 


diminished by extrinsic rewards for writers, 421 


four-stage model, 814 
Lehman’s tables and graphs concerning some form of, 320 


modern emphasis on expertise in, 820 


political environments nurturing, 318 


promoting team creativity, 516 
requiring thinking outside of the box, 832 
restructuring as the basis for, 817—819 


source of, 17 

transfer of expertise and structured imagination in, 828 
“credentialism,” 132 
credibility, in TMS, 512 


credit card transactions, fraud detection software for, 97 


cricket 
batsmen in, 685, 68 
bowlers in, 679, 749 


criteria, for case studies, 295 


N 


criterion 


performance, 216 


as relatively narrow, 223 

validity, 215 
critical causes, difficult to isolate in cases, 292 
critical cues, 680, 725 


“critical incident” descriptions, 457 


critical period, in the acquisition of chess skill, 609 
critical reflection, capacity for, 37 
critical thinking, 782 


crop dusters, societal expectations for, 109 
cross-cultural comparison, benefits of, 435 
cross-lagged’-design, 262 


cross-sectional designs, of historiometric studies, 315 


cross-sectional study 


comparing different population groups, 277 
preceding a longitudinal study, 271 
typical, 277 


cross-sectional time series analysis, 316 


cross-training, leading to shared team interaction, 513 


cross-validation, performing, 265 
crystallized intelligence (Gc), 227 


changing how problem-solving is carried out, 607 
defined, 775 
described, 836 


human capacity for accumulating, 273 


testing for, 488 


“crystallized” or “pragmatic” aspects, of intelligence, 838 


CTA. See cognitive task analysis (CTA) 


cue—hazard relation, research focusing on, 363 


cues 
activating appropriate goals and models, 717 
critical required to lead to a match or a near match, 722 


identifying perceptual, 681 


as often dynamic, 455 


salience of, 716 

understanding which are important, 386 
cultural activities 

expertise and, 436 

hard to change, 435 

learned implicitly, 435 

teaching as one, 434-435 


cultural assumption, that teaching is not something generally subject to 


improvement, 445 
cultural differences, anong members of a team, 517 


cultural diversity, of virtuality, 517 


cultural environment, promoting the development or neglect of skills, 544 


cultural factors, central to both the development and practice of expert 
teachers, 434 

cultural knowledge, influencing military personnel during crowd 
management in the Middle East, 465 


cultural lenses, breaking free of, 435 
“cultural modernity,” 45 


cultural nature of teaching, raising problems for educational change, 435 


cultural routines 
changing, 443, 444 
of teaching, 435 


cultural sense-making, 465 


cultural-historic activity theory, 106 


culture(s) 
adapting to an ever-changing environment, 435 
of humans as complicated and sophisticated, 40 


“current condition,” 444 


current knowledge state, assessing for students, 440 
current routines, building awareness of, 444 


curricula, as means of organizing learning, 118 


curriculum 
knowledge, 441 


original meaning of, 117 


strategies, 538 


curvilinear function, describing the output of creative products, 320 


CVA (Classroom Video Analysis) measures, as better predictors of student 
learning, 441 
CYC, encoding common sense knowledge, 94 


cycles, involving analysis, planned activity, and assessment, 448 


cycling, between goal-driven and data-driven processing, 717 


da Vinci, Leonardo. See Leonardo da Vinci 
daily experiences 
deliberate practice (or deliberate performance) during, 447 


mediation of, 113 


daily training logs, 266 
D’Alembert, Jean Le Rond, 6 
dancing, series of studies on the AON, 246 


Darwin, Charles, on zeal and hard work, 629 
Dase, Zacharias, 626 
data 


collection and analysis, 261 


combining new to build understanding, 718 


integrating all relevant into a “story,” 466 


needing theories, 795 
recognizing patterns and anomalies in, 467 
data-directed problem-solving, as forward chaining, 99 


Data/Frame (D/F) Model of Sense-making, 461-462, 463 


daydreaming, conducive to writing, 422 


de Fermat, Pierre, 479 
de Groot, Adrian D., 62 
analysis of “think aloud” protocols, 701 


chess masters picking a promising move, 820 


experimental studies, 597 


inviting chess masters and skilled club players to “think aloud,” 11 
study of expert performance in the study of chess, 200 
debriefing 


improving team performance, 509 


scheduling after performance episodes, 523 
decade of intensive preparation, required to achieve excellence, 425 
decade of practice. See also “10-year rule” 


progressing from knowledge telling to knowledge transforming, 423 


decentralized leadership, 509 
decentralized teams, 459 


deception, 247 


decision(s) 
analyses for high-stakes decision making, 480 


approximating normative standards, 483 


compared with judgments, 476 


emerging from recognition, 453 


explanations of, 195 


exploring cognitive dimensions made by experienced workers, 175 


making effective, 398 


making in complex situations, 453 


making incrementally and iteratively, 455 


most as ill-structured, 480 


requiring particular kinds of expertise, 26 


as response to naturally occurring situations, 758 
superior defined by optimization analyses, 479 
decision aids (visual aids), 366, 496 
decision competency assessment, 486-488 
Decision Ladder(s) 


identifying cognitive states and cognitive processes, 167 
as the product of Work Task Analysis, 179 
suite of, 178 

decision makers. See also decision making 


balancing risks and implications, 498 
flow of information to, 174 


least cognitively “able” among the most skilled, 482 


in sport, 660 


decision outcomes, independent of intelligence, 488 


Decision Outcomes Inventory (DOI), 486 


decision paths, observed, 179 


decision performance and ability structure, 486 


decision points 
employing CTA to identify, 458 


having a small set of options, 185 


decision problems, solving, 405 


decision processes, identifying, 179 
decision quality, predicting, 478 


decision research, in chess, 481 


decision rules, applying, 486, 487 


decision science, 478 

decision sciences and technologies, 498 
Decision Skills Training (DST), 459 
decision strategies, 166, 479 


decision support 
simple, powerful, 494—497 
systems, 291 


technologies, 480 


Decision-Centered Design (DCD), 170, 466 
decision making. See also decision makers; general decision making; 


judgment; skilled decision making; superior decision making 
across cultures, 465 


advice-giving tasks and, 97—98 
central issues of, 187 
in chess and other domains, 748—749 


from deliberate evaluation and representative understanding, 496 


effective, depending on high levels of SA, 715 


effective, involving consideration of options, 493 


efficient processing of object features and, 585-586 
expert, 464465, 477 
by experts and non-experts, 476 


forums for, 22 
heuristics, 394, 479 
improved quality of, 96 


individual differences in, 486 
as a knowledge-centric and comprehension-oriented activity, 493 


mapping generic sub-tasks involved in, 179 


naturalistic studies of, 166 


not limited to the intuitive, 460 
reckoning with risk and uncertainty, 493 


requiring reasoning and metacognitive skills, 478 


as the result of a team effort, 458 


systems available to support, 719 


uncertain, missing and conflicting information as common, 730 


unpacking the “front end” of, 456 


decision making performance 


on naturalistic risky decision making tasks, 492 
of skilled basketball players, 686 
decision making skill, 476-499 
as acquired, 497 
as assessed by ADMC, 488 


measuring, 493 


numeracy as the strongest single predictor of, 492 


psychometric studies of, 486—493 


deck of cards, in an Ask sequence, 404 
declarative knowledge 

developing, 114 

of experts, 455 


individuals first requiring, 115 
declarative or procedural learning, as either positive or negative, 115 


de-contextualization, of tests, 849 


deductive logic, 479 


deductive reasoning, 377 
deep learning, 93 
in a changing world, 817-819 


requiring some element of struggle, 439 


taking time, 440 
deep level, of team composition, 515 
DeepBlue chess program, 86 
“deeper (more principled)” encoding, of domain-related information, 66 
deeper representations, characterizing expertise in a domain, 68 
deep-level composition variables, meta-analysis on, 515 


“de-expertise,” plasticity-induced loss of skills as, 563 


delayed onset (late bloomers), careers peaking later, 321 


deliberate, goal-directed activities, 43 
deliberate performance. See also performance 
concept of, 448 


daily classroom experiences as a site for, 447 
defined, 446 


experts in diverse domains engaging in, 454 


as the most feasible route to expertise, 446 


notion of, 398 
deliberate play, 264 


deliberate practice. See also purposeful practice 
in abacus training, 626 


accounts by older experts, 840 


accumulated by older amateur pianists, 846 
accumulating, 276 


activities for different musical styles and sub-skills, 538 


amount undertaken during many years, 551 


aspects of not present in other types of training activities, 757 
carrying out successfully, 755 


characteristics of, 424 


chess performance, strong predictor, 609 


chess players, time spent in, 608 
conditions for in teaching, 445—446 


conducting alone or with a group or team, 655 


creating a forum for within undergraduate medical experience, 338 


critical in the development of adaptive expertise, 445 
critical in the development of expertise, 831 
definitions for, 654, 756, 762 

described, 480, 536, 551, 831 


as a designed experience, 445 
detecting weaknesses and developing existing or new skills, 844 


developing entrepreneurial expertise, 397—400 


in different domains of medicine, 331 


differing for particular individuals, 763 
effective for relatively more knowledgeable learners, 807 


effective way to change (improve) performance, 764 


effectiveness in attaining elite and expert levels of performance, 75 
engaging in, 76, 459 


expert performance requiring, 43 


for expertise maintenance and in later adulthood, 842—843 
feedback to drive, 341 

as goal-directed optimized practice, 538 

histories, 265 


on identifying cues presaging hazards, 368 


identifying features at the micro-level, 656 


increasing the complexity of a task, 343 


individualized practice as more effective, 760 


institutionalized, “schooled” activities and, 106 


involving conscious concentration on the skill, and informative feedback, 
445 
issues with the original definition of, 655 


as just one type of practice, 764 


juxtaposed to mere experience, 365 

as the key factor in the acquisition phase, 842 
lesson study as a lab for, 446—447 

likelihood of dropping out, 609 


long-distance race of, 837 


mechanisms initiating and maintaining, 107 


mechanisms mediating experts’ superior performance, 14 


as the most effective practice, 761 


not inherently enjoyable and not offering any immediate reward, 654, 655 


not permissible during real-life clinical procedures, 341 
performed within work contexts, 397 


versus practice, 345 


practice activities as, 755 


premise of the theoretical framework of, 344 


as a prerequisite for attaining excellence, 551 


as quite challenging, 480 


rarely possible for medical students, 336 


refinement of the concept of, 551 
required to create complex artifacts, 46 


requiring supervision from a trained teacher, 389 


role in the development of expertise, 64 
secondary competencies and, 43 
supervised and designed by a teacher, 398 


sustaining struggle and connections over time, 439 


tending to produce remarkable differences, 480 


as working hard at hard problems, 464 
deliberate practice model, 839, 847 
deliberate practice (the micro-level), 656—658 


deliberate practice theory 


applicable in a variety of fields of expertise, 276 


in the domain of sport, 655 


practice activities circumventing current limitations, G09 


testing key predictions of, 657 


deliberate reflection procedure, 339 
deliberation 
interplay with intuitive judgments, 74 
in the light of values, 498 
DeLillo, Don, 426 
delusions, distinction with illusions, 591 
Deming, W. Edwards, 446 


democratic control, of the activities of experts, 139 


democratization, of expertise, 22 


demonstrated performance, of an expert, 127 


demonstration projects, 297 
DENDRAL research project, 87, 88 
dendrites, growth of new, 553 


dentists, learning structured patterns, 153 


dependent samples tests, 279 


depiction, in a given medium, 584 


depictive accuracy, operationalizing, 590 
depictive skill, 590 
depth 

of knowledge, 639-640 

of planning, 201 

of search, 602, 605 


depth-first approach, of novice behavior in problem-solving, 384 


depth-first explorations, of solution concepts, 384 
derivative knowledge, 648 


descriptions, versus prescriptions, 265—266 


descriptive project, 478 


descriptive studies, 295 


design 
ability, 372-373, 375 


activity in right dorsolateral prefrontal cortex, 376 


as a co-evolution of both solution and problem, 378 


decisions, 379 

developing expertise in, 384—385 
documents developed by an expert, 291 
effort, quality of, 372 

emergence of as a profession, 373 
individual, 316 


inherent within human cognition, 372 


intervention, 180 


key aspects of expertise in, 376-382 
protocol study, 377 

representative, 679 

role of, 401 

sensitivity to expert knowledge, 187 


setting and changing goals inherent elements of, 386 


solutions, 382 


strategies for within Cognitive Systems Engineering, 170-171 


strategy guiding the transition from analysis to design, 170 


students, changes in individual behavior, 385 


study of expertise in, 373 


task involving the layout of furniture within a conference room, 376 
toolkit, 170 
understanding expertise in, 373—376 

The Design of Experiments (Fisher), 485 

designers. See also expert designers 


adopting a conjectural approach, 377 


changing goals and constraints, 378 


dealing with ill-defined problems, 386 


finding a partial structure, 378 


jumping to ideas for solutions, 378 
modes of activities of, 379 


moving rapidly to early solutions, 377 


novice, 373 


“opportunistic” behavior of, 383 

pursuing only a single design proposal, 382 

using knowledge to precedents abstracted into solution chunks or 
“schemata,” 381 


watching at work, 374 


designing 


conducting by purely internal mental processes, 380 


not a strictly hierarchical process, 381 


as not “normal” problem-solving, 375 
as a Shared, social process, 374 


desirable system state, identifying, 180 


desired outcome, musicians imagining, 541 


“detachment,” in professional work, 133 
determinants, of initial task performance versus final task performance, 221 


development of expertise 


during adolescence, 276 


conclusions about how best to support, 121 


stages of, 385 
development portfolios, 807 


developmental disorder, in the acquisition of numerical concepts, 617 


developmental history profiles, of cricket batsmen, 687 

Developmental Model of Sports Participation, 688 

developmental or educational process, leading toward expertise, 65 
developmental theories, proposing differentiation of abilities across time, 607 
Devi, Shakuntala, 619 


deviations 


from the “best” bottom-up depiction, 588 


by expert teams classified as innovations, 510 


device or process, malfunctioning, 96 

devices and systems, diagnosing and troubleshooting of, 96 
Dewey, John, 437, 783 

D/F (Data/Frame) model, 461—462, 463 


diagnoses 


accuracy of increased as a logarithmic function of the number of 


mammograms, 204 


acquiring multiple exemplars of, 338 


expertise in, involving multiple kinds of knowledge, 338 


first generating and then generating an explanation and rationale, 205 


generating alternative, 339 


generating becoming more efficient, 205 


impact in training of instructions, 783 


process constructs of underlying, 779 


processes associated with superior accuracy in, 70 


diagnosis-related knowledge (“reflection”), impact on reducing error, 336 
diagnostic accuracy 


higher among groups trained using the contrastive approach, 783 


inversely related to time, 335 


lower in the serial-cue than the whole-case format, 338 


measuring individual differences retrospectively, 203 
diagnostic errors, 332, 333, 336 


diagnostic performance, in internal medicine, 344 


diagnostic tests, in medicine, 203 


diagnostic thinking, differences as a function of expertise, 205 
diagnosticians 


as error prone, 333 


expert, organizing diagnostic hypotheses, 69 
diagrams, usefulness of, 90 
“dialectics of sketching,” 380 
Diamondi, 700 
diaries/training logs, 259 


dichotomous variable, virtuality as, 516 
Dickin Medal, 50 

dictation scores, 636 

Diderot, Denis, 6 

diet coaches, 139 


difference variables, 298 


differential expertise, area of the measurement of, 312 


differential preservation account, 839, 841 
Differentiated Model of Giftedness and Talent, 272 
differentiation, 127, 239 

diffusion tensor imaging (DTT), 235, 559, 560 
digit span, measurement of individuals’, 199 

digit span task, 698, 701 

digits, encoding and storing in LTM, 199 
Digit-Symbol Substitution tests, 848 


dimensions, of expertise, 27—28 


diminishing returns, from purposeful practice later in life, 843 
direct guidance, in the workplace, 117 
direct hazards, detecting, 363 


direct observation 
of clinical judgment as minimal, 779 
employed in case studies, 293 


“directions,” in a heuristic at a general level, 821 


disability in mathematics, as a marker for low intelligence, 619 


“disciplinary identity,” of scientists, 137 
disciplines, in the sciences, 137 
discourse 


appealing aspects of, 131 


organization, 645 


of professionalism, 128, 130-131 


discovery learning, opportunities for, 659 


discretionary time, 726 


discriminant analysis, 279 
“discriminant validity,” 215 
diseases 


actors trained to simulate, 332 


diagnosing causes and recommending therapies for infectious, 85 
exemplars of, 337 


expert physicians representing, 66, 71 


human biology producing, 338 
knowledge of the clinical presentation of, 339 
numeracy scores and, 477 


pathophysiological knowledge of, 337 


remembering particular by referencing patients, 120 


dispositions, 24, 112 


dissociation, between thinking skills and perceptual-memory skills, 597 
distractions, creating a significant challenge for SA, 724 
distributed networks, 156 


distributive property, of multiplication over addition, 620 
distributive structure, fluent recognition of, 620 


diversity, among workers, 510 


division of labor, in societies, 137 
DJ, practice for, 544 
DLPFC (dorsolateral prefrontal cortex), 238, 240, 243 
DNA, 823 
doctors. See medical doctors 
Dodge, R. Wagner (“Wag”), 819 
insight as a sudden flip of understanding, 819 


knowing about backfires, 830 


using fire to create a protective buffer zone, 830 


dog experts, remembering and recognizing images of dogs, 158 


dogs (Canis lupus familiaris) 
breeds perceived to be intelligent, 52 
competing in athletic activities or sports, 51 
excelling at a wide variety of tasks, 50 
genetic components for wrestling and fighting, 52 
improving markedly with practice and repeated exposure, 51 
with longer legs running faster than dogs with shorter legs, 52 
domain(s) 
attaining the highest level of performance in several, 67 
distribution of in IAAT Papers, 98 


with enormous individual differences for expert performance, 44 


examined by NDM, 453 
of expertise, 706—707 


increased experience associated with superior memory performance, 699 
of knowledge, 34, 108 
in knowledge-based systems, 85 


length of experience unrelated to improvements in professional 
performance, 75 


with public, fair competitions, 746 


puzzle of, 143 


resembling chess, 203 


SA transfer between similar, 724 


specified for a case study, 293 


studying specific to understand distinctive heuristics and cognitive 
processes, 394 

of successful applications of expert systems, 96 

talking about, 53 

types of decisions, 142 

of writing, 413 

domain effects, within the psychology of expertise, 142 


domain expertise 


expanding the study of, 463-464 


of professional writers, 419 


domain experts 


in aviation, 736 


rarely considered more than one option at a time, 453 


using long-term memory as a kind of working memory, 420 
domain knowledge 


capturing in terms of what to know, 85 


compensating for ability shortcomings, 223 
as an explanation for the superiority of expert performance, 541 
domain managers, 27 


domain of achievement, 313 


domain of practice, improved performance restricted to, 344 
domain specificity, 396, 578, 623, 719 


domain-general abilities, as a function of level of accomplishment, 836 


domain-general functions, in tasks measuring older experts, 841 


domain-general problem-solving, as limited, 11 
domain-knowledge tasks, approach to understanding, 229 
domain-related activities, effects on attained performance, 17 
domain-specific acquired patterns and associated actions, 67 
domain-specific actionable cognition, 772—774 
domain-specific capacity, in numerical abilities, 617 


domain-specific experts, 679 


domain-specific information, imposing a working memory load for novices, 
797 
domain-specific knowledge, 89 


allowing experts to escape constraints on working memory, 419 


in declarative statements, 89 

enabling quick orientation in a new situation, 235 
of experts, 455 

of objects, 584 


overriding developmental differences, 606 


representing, 88 
stored in LTM, 234, 248 
domain-specific knowledge and schemata, details of artists remaining under- 


characterized, 592 


domain-specific knowledge and skills, increasing relevance demonstrated for 
older professionals, 836 
domain-specific mechanisms, supporting expert performance at any age, 847 


domain-specific memory, 235 


domain-specific reproducibly superior performance, 746 
domain-specific rhetorical skills, years needed to acquire, 423 


domain-specific role models, availability of, 318 


domain-specific rules, 89 
domain-specific skills and knowledge, 318 
domain-specific stimulus patterns, 681 


dominant left hemisphere, 415 


dopamine (rewardng experience), 553 


dorsal stream area (SMG), activation of, 243 

dorsolateral prefrontal cortex (DLPFC), 238, 240, 243, 555 

dots, selecting the larger of two arrays correlating with arithmetical expertise, 
619 

double helix of DNA, discovery of, 823, 824 

double take of expertise, 248 


double-digit numbers, algorithm for multiplying, 623 
Dragon’s Den (TV show), 399 


dramatists, lifetime output of eminent French and English, 311 


drawing 


conceptual issues, 590—592 


as the end point of the design process, 380 


experience, 589 
expertise, 5/76—593 


explanations for ability in, 580 


future directions in expertise in, 588—592 


methodological issues, 588—590 


perceptual processing and, 583 


prodigies, 592 


savants, 579 


shifting attention between different modes of processing, 586 
tasks, 578 


versus writing task for university professors, 421 


drawing accuracy 
assessing, 582, 590 


multiple dependent measures of, 589 


drawing errors, misperception hypothesis of, 581 
drawing skill 


discriminating visually presented real versus nonsense words, 586 


measuring, 582, 589 
operationalized by objective errors in copying a photograph of a house, 
586 
variability in attributed to differences in visual perception, 576 
drawing studies, studies treating subjective accuracy ratings holistically, 590 
“The Dream Team,” 506 


“dream team” status, approaching, 523 


dreaming at night, conducive to writing, 422 
Dreyfus, Hubert, 26 


drivers. See also experienced drivers; expert drivers; less-experienced 
drivers; non-drivers; vehicle operators 


autonomous systems redirected attention to secondary tasks, 735 
braking task, 362 


experience in change detection, 359 


experiencing an increasing sample of driving situations, 363 


giving more attention to detecting potential traffic hazards, 722 
licensed racecar, 358 


more experienced, actively modified the situation, 360 


proficiency level, 357 


redirecting attention to other tasks, 734 
training programs, 365 


young benefiting from driver support systems, 367 


driving 
accident rates and traffic violations decreasing with experience, 356 


attaining acceptable performance, 752 


people with better SA performing better, 725 
SA and expertise in, 734—735 
skills like, 222 


ubiquitous in modern society, 734 


drop-outs 


improving more slowly over time in chess, 609 


from longitudinal study, 271 
risk of bias in case of, 281 


drugs, altering consciousness for writers, 422 


“dual process” model, of reasoning, 333 


dual processing theory, describing thinking strategies and memories 
retrieved, 333 

dual-process theory building, in psychology, 336 

Dunker, Karl, 193 

dynamic decision making, model of SA in, 716 


dynamic environments, 356, 682 


“dynamic stereotypes,” 565 


dynamic team state, 508 


dynamic video images, 682, 683 


dynamics of knowledge, 138 


dyscalculia, 627 


dysfunctional genetic predisposition, 552 


dysphoria, characterizing writer’s block, 422 
dystonia. See also focal dystonia; musician’s dystonia 
family history of, 565 


patterns of, 563 
triggers of, 567 


early and late occlusion activation, differentiation between, 247 


early career commencement, 320 


early experiences, affecting later perception, 151 


early humans, possession of expertise in skills, 45 

early life experiences, controlling for animals, 52 

early mature, 276 

The Early Mental Traits of Three Hundred Geniuses (Cox), 312 
early musical training, stabilizing the sensory-motor system, 569 


early occurring (advance) information, experts better able to pick up, 661 


early (pre-contact) occlusion conditions, experts’ performance levels above 
chance, 661 
early predictors of talent, attempting to identify, 666 


early sensitive periods, of musicians, 552 


early trained musicians, brain changes in, 561 
Ebbinghaus illusion, 583 


ECG. See electrocardiograms (ECG) 


ecological concepts of expertise, 784 


ecological demands, coping with, 42 


ecological dominance and social competition theory (EDSC), 40, 41 


ecological fallacy, 280 


ecological perspective 


assessing expertise from, 776—777 


rethinking fundamental issues, 771 


ecological risk literacy, 489, 490 
ecological selection, controlling, 41 


ecological view, of expertise, 771—772 


ecologically based approaches, focusing on everyday competencies and real- 
life expertise, 849 


ecologies, coping with disparate, 41 


economics journals, rhetorical style employed, 419 
Edison, Thomas A., 296, 829, 830 
editing 


as complex, 414 


dissociating the author from, 418 


education. See also training 
of calculators, 625-626 
in design, 384 


developing clinical reasoning, 336—337 


genius and exceptional talent associated with, 317 


lack of a consensus on the aims of, 431 
measuring students’ performance on standardized tests, 747 


preparing students for specific occupations, 114 


research in a particular domain, 65 


educational and occupational training programs, design of, 291 


educational contexts 
knowledge of, 441 


perceiving structure in, 443 


educational discourses, questioning, 108 
educational institutions, programs and instruction in, 113 


educational principles, using in chess training, 606 


educational psychology, instructional design and, 64—65 
educational restructuring problem, 297 

educational settings, developing occupational capacities, 109 
educational systems, enabling deliberate practice, 46 


“effect,” predicated upon events and decisions, 401 


effect sizes, 280 
effective leadership, 298 


effective learners, 121 


effective learning, insights into, 64 
effective moves, retained, 796 
effective practice 
as being less constrained, 396 
instruction and, 667—668 
effective teams, 507, 509 


effective workers, 166 
effectual Ask, 400 


effectual heuristics, 394 


effectuation 
heuristics termed, 394 


internalizing “complex indeterminate causation,” 394 


practice in, 398 
rejection of prediction in the decision making heuristics of expert 
entrepreneurs, 400 


summary of research, 394—396 


efficacy, virtual team performance and, 518 


efficiency 

imperative of, 122 

increased through pruning, 245 
effort 


investing in practice, 538-539 

not directly observable, 538 
effortful practice, 45, 396 
effortful processing, 116 


eighth grade mathematics teachers, following a common script in the United 
States, 434 
Einstellung (set) effect, 604 


Einstellung chess position, example, 604 


“either—or” perspective on expertise, 784 


electrocardiograms (ECG) 


approaches to interpretation, 339 


increased accuracy with higher levels of expertise, 204 


training on reading, 783 
electroencephalography (EEG), 236 


electromyographic recordings, in patients suffering from hand dystonia, 567 


electronic development portfolios, assisting learners in choosing new learning 
tasks, 807 
electronic health records, problems with, 171 


electronic systems, reducing ambiguity inherent in, 466 


electronic troubleshooters, 186 


electrophysiological methods, 558 


elephant, images of in a limited-line tracing task, 585 


elicitation methods, need for systematic, 168 


elicited knowledge, organizing, 187 


Eliot, T. S., 425 
elite(s), 22, 139 
elite athletes, 654, 655, 660 


elite cricket batsmen, 685 


elite endurance runners, 73 


elite female players, testing for interval endurance capacity, 285 


elite musicians, 751 


elite performance 
experience and training required for, 76 
functioning as a costly signal, 44 

elite performers 


avoiding the arrested development associated with automaticity, 334 


engaging in relevant training activities early, 13 
elite soccer players, 658 
elite tennis players, 72 
elite triathletes, 655 


elite view of expertise, leaving interactional experts unappreciated, 139 


elite wrestlers, 655 


elite youth soccer players, 277 


Elo rating, of chess players increasing as a function of hours of practice, 344 


Elo rating system, of chess tournament performance, 597 
embedded explanations/prompts, vs. text only, 799, 801 
embodied cognition, theme of, 591 

embodied knowledge, 115, 121 


embodiment 


expertise and, 28 
language and, 26 


embouchure dystonia, 567 


emergency medicine, explicating expertise, 65 
emergency planning director, technology and, 174 


emergency response teams, 173-174 


emergency situations, during flying, 760 


emergent leadership, 509 


emerging methods, for exploring cognitive work, 182—185 


eminence 


attained by artists or scientists as a function of teachers and mentors, 315 


based on the amount of space devoted to in standard reference works, 314 


for classical music composers, 319 


recorded by space allotted in reference works, 314 


eminent achievers, tending to come from distinguished family pedigrees, 317 


eminent individuals, often the offspring of a small number of families, 10 
eminent men, with an unpromising history, 618 


eminent people, tending to have eminent parents, 618 


emotion(s) 
processed in structures of the limbic system, 554 
relation with reason, 332 


role of in reasoning, 347 


emotional and performance skills, brain areas supporting, 553 


emotional blocks, 420 
emotional challenges, managing, 420—422 


emotional connections, managing with students, 442 


emotional cues, 464 


emotional responses, identifying potentially intrusive, 464 
emotional state, regulating the writer’s, 421 


emotional ups and downs of writing, 422 


emotion-related brain areas, 554 


empirical designs, 403 
empirical generalizations, 358 


empirical indicators, separately recorded and analyzed, 198 


empirical studies, using physiological sensing and wearable computing 
technology, 518 


employability, securing and sustaining, 107 


employment and entrepreneurship, choice between, 390 
“EMYCIN” (“Essential MYCIN”), as the first expert system “shell,” 89 


enabling conditions, for a disease, 337 


encapsulated knowledge, reorganizing, 337 


encapsulation theory, 106 


encoding. See also selective encoding 
interactive form of, 684 
Strategies, 7/04 
enculturation or apprenticeship model of learning, 23 
Encyclopédie (Diderot & D’Alembert), assembling all available knowledge 
in, 6 
endorphins (joy), 553 


endurance athletes, 248 


endurance sports, age of peak performance, 273 


enemy, detecting information about, 733 


enemy forces, practicing denial of information, 730 


engaged time, amount of for students, 433 


engagement 
in the lived experience of work, 119 
in play versus practice versus competition, 668 
relations between amount of and attained improvements of performance, 
#o3—/51 


training, 365 
engineered systems, diagnosis of, 96 
engineering branch, of AI, 86 


engineering design, as a social process, 374 


engineering fidelity, 341 
engineers 


computer-based training program for novice, 458 


experienced making a preliminary evaluation of tentative decisions, 384 


fixated in the traditional sense, 383 


limited sensitivity to cognitive issues, 170 


studies of expert electronics, 384 
English language 
54,000 word families, 634 
largest vocabulary of any known language, 634 


mastery of the complete lexicon of beyond native speakers, 634 


maximizing the learner’s exposure to, 647 


often vague, 100 
English Vocabulary in Use application, 648 
English-speaking country, spending time in, 647 


enjoyment 
as the least important feature of “deliberate practice,” 656 
sport-related differences with respect to, 655 


ensemble characteristics, 153 


entanglement, of expertise studies with history, 66 
entrepreneur(s). See also expert entrepreneurs 
accumulating knowledge through experience, 391 


cognitive frameworks used by experienced, 392 


complex and uncertain situations as the domain of, 400 


developing heuristics to deal with uncertainty, 394 
developing the script-scenario instrument, 392 
global impact of, 391 


improving performance, 393 


learning vicariously or transferring skills learned via practice in other 
domains, 398 


novice starting with tentative Asks, 400 


origin of the term, 390 


path to becoming as not special but general, 397 
as risk avoidant, 390 


specific practicable cognitive activities of, 397 


targeting angel investors or venture capitalists, 399 


varying on numerous important dimensions, 404 
entrepreneurial domain, 396 
entrepreneurial expertise 


deliberate practice in the development of, 389-405 


heuristics minimizing or eliminating reliance on prediction, 394 
reviews of, 206 
studies of, 392—394 


entrepreneurial learning, performance impacts of, 393 


entrepreneurial personality profile, decoding, 390 
entrepreneurial process, 398, 399 


entrepreneurial scripts, 397 


entrepreneurial teams at MIT, 519 
entrepreneurship 
as an academic domain, 391 


as Cco-creative, 401 


as a driver of jobs and economic development, 391 


proficient performance in, 394 


purposeful practice in, 392, 396-397 


researchers in the “field” of pre-occupied with a perceived lack of 


legitimacy, 391 


uncertain yet human domain of perceived control, 395 
Entrepreneurship, Inc. company, 406 


entrepreneurship research, 390-391 


entwinement 
of (aspiring) experts with their world, 35 
of expert knowledge and experts, 34 
of persons with their world, 34, 38 
environment 
cycle of engaging, 771 
key features of affecting SA, 719 


recognizing the numerosity parameter, 617 


for writers, 421 


environmental contribution, to musician’s dystonia, 566 


environmental cues, 775, 777 
environmental factors 


implications on aircraft dynamics and behaviors, 728 


resulting in communication breakdowns during “handover,” 347 
environmental fidelity, 341 
environmental information and past experience, stimulating people’s use of, 
782 
environmental organizing and linking principle 
justifying preceding principles, 797 


reducing working memory load, 805 


retrieval using, 802 


environmental services, 133 


environmental signals, switching genes on or off, 797 
environmental variability, 41 


environments, conducive to tacit-knowledge acquisition, 783 


epigenetic system, 796, 797 
episodic LTM regions, 240 


episodic memory, 240 


“epistemic action,” 775 


epistemic communities, professions as, 132 


epistemic cultures, 132 


epistemic injustice, limits of attribution and, 22—2 


epistemic “objects,” 133 


“epistemification,” 134 
epistemological emphasis, of research on expertise, 35 


epistemology, of professional work, 132-134 


equations, solving simple, 620 


equivalent current dipole strength, 557 
Eriksen flanker task, 156 
ERP patterns, during tactile perception tasks, 842 


error Management procedures, 291 


error reduction, 334 
error-free analysis, 493 


errors 


arising from biases, 334 
as a consequence of heuristics or knowledge deficits, 334 
in the diagnostic process, 333 


by experts in contrast to novices, 291 


types of, 332 


escape fire 
setting now taught to all smokejumpers, 819 
Wag Dodge’s, 819, 830 


esoteric skills, expertise and, 27 


esotericity, 27 
essays 


of college students as freshmen compared to juniors, 417 


of college students contrasted with professional academic writers, 417 


essential knowledge elements, abstracted, 170 


essential skills, promoting resilient and adaptive decision making, 483 


estimated engagement, 759 
estimation, 446 
ethics, 780 


ethnographic survey studies, of professional writers, 424 


“Etudes,” for guitar and for piano, 544 
Euclid, 134 


European vernacular musicians, 544 


evaluation, 277 


anxiety, characterizing writer’s block, 422 


of how experts structure their days, 258 


resulting in team adaptation, 511 
events 


assessing the importance and severity of, 724 


nature of, 295 


permitting application of advanced analytic procedures, 304 
reflecting an action, or set of actions, 292 


specifying legitimate and illegitimate, 293 


systematic identification of, 294 


everyday activities, goal for, 753 
everyday conversation purposes, requiring 2,000—3,000 word families, 637 


everyday expert performance, accounting for, 770 


everyday expertise, conceptualizations of, 771 


everyday goal-directed activities, engaging in, 115 
everyday learning activities, 108 


everyday perception, distinction with artists’ expertise, 577 


everyday problem-solving, numeracy research and, 477 


everyday skills, relatively easy to acquire, 75 
everyday work activities, 121—122 
everyday world, encoding facts about, 94 
evidence, assessing the strength of, 91 
evidence-based medicine, 347 
evolution 
of the expert advantage, 680-681 
of expertise, 40—46 
evolutionarily novel domains, 40 
evolutionarily novel ways, for developing expertise, 46 


evolutionary biology, 795 


evolutionary considerations, for expertise and other cognitive phenomenon, 
93-54 
evolutionary educational psychology, Geary’s, 795 


evolutionary theory, using, 795 


examples, modifying the search strategy, 383 


excellence, depending on deliberate practice, 142 


exceptional achievement 


developmental antecedents of, 317 


examining across the entire life, 313 


exceptional creators, less likely during times of political anarchy, 318 
exceptional development, understanding the nuances of, 257 
exceptional individuals, as gifted, 67 


exceptional mathematical abilities, early reviews of, 616 


exceptional performance 
expertise as, 49—50 


gathering information on, 192 


relying on well-honed procedures, 114 


exceptions, to rules, 24 
executive control, detrimental when dealing with problems requiring creative 
thinking, 816 
executive functioning 
lack of in creativity, 816-817 


positive role in solving problems with insight, 832 


executive functions, benefits of physical fitness for, 847 


exemplar model, of reasoning, 334 


exemplars, 333 
exercises, for professionals on the job, 446 


expanded working memory, 701, 708 


expectations 


managing in teams, 509 


playing an important role in SA, 718 


expected values 


explicitly calculating, 482 
selecting options with, 479 
experience(s) 
accumulated, having limited effects on attained level of performance, 
792-754 


accumulation of increasingly complex chunks and pattern—action 
associations, 76 

alone, as insufficient for development of expertise and expert performance, 
365 

augmenting the workplace experiences, 119 

broad dimensions to support and augmentation of, 117 


complex relationship with expertise, 347 


from concrete work playing into a profession’s body of knowledge, 133 


connecting otherwise different objects, 364 


effects with age controlled, 357 


enriching to be more pedagogically effective, 119 

of an expert coloring articulation of knowledge, 94 
extensive episodes of promoting adaptability, 111 
improving situation awareness and hazard perception, 363 


increased having a small effect on the accuracy of clinical judgment, 779 


increasing fixations, 364 


leading to the development of better strategies and better mental models, 
362 


as the main variable used to indicate expertise in teaching, 436 


in making art, 578 


meta-processing varying with, 365—366 
new knowledge and skills gained via, 393 


not guaranteeing the development of expertise, 436 


not sufficient for the development of expertise and expert performance, 75 
openness to, 391 
perceptual, 151, 157-160 


permitting honing and linking of concepts and procedures, 116 


as a proxy for expertise, 436 


relationship with hazard perception, 362 
representative compacting or accelerating effortful, 76 
role in human expertise development, 54 


structuring to assist in occupational activities, 117 


in transportation, 357 
as a Vital part of the transformation to expert, 385 
in workplaces and across working life as key sources of occupational 


expertise, 108 


in workplaces providing essential learning opportunities through action, 
114 


experience levels, for problems with SA, 727 


experience sampling methodology (ESM), 403-404 


experience-based learning, methods facilitating, 783 


experience-based recurrence reduction, 346 


experience—performance relationship, as weak, 393 


experience-specific patient experience, direct measures of, 346 


experienced designers. See expert designers 


experienced drivers. See also drivers; expert drivers 


adjusting scanning patterns to the road type, 734 


changing behavior to adapt, 359 


classifying movies of driving, 364 


commiting fewer driving infractions, 366 


controlling a powerboat simulator, 359 


deprived of the ability to use their knowledge, 364 
drove faster in clear conditions, 359 
looking further ahead of the vehicle than lesser drivers, 686 


maintained better lane control in a driving simulator, 367 


operating in very familiar environments, 723 


outperforming non-drivers when allocating attention as they would be 
required to do while driving, 359 
processing cues and converting cognitive portions of driving to a level of 


automaticity, 723 


reacted faster to hazards than novice drivers, 734 
reduced speed because of fog, 359 


responding on the cue—hazard contingencies on the road, 363 


superior at (some types of) hazard perception, 363-364 


experienced individuals, encountering complex or rare situations, 74 


experienced pilots. See also pilots 


altering pitch in black hole conditions, 360 


challenged to process information in working memory, 724 


focusing on planning and preparation specific to the flight, 726 
with high SA, 728, 729 

knowing about a passing aircraft, 721 

with moderate SA, 728, 730 

more likely to change their behavior to adapt, 359 


using the new ASAR technology, 367 
experiential evidence, supporting associations, 91 


experiential nature, of tacit knowing, 772 


“experiential—educative—acculturation influences,” 775 
experimental approaches, 260 


experimental methods 


applied to artificial projects, 373 
defined, 135 


experimental school, visions and plans for leading evaluated, 298 


experimental studies 
of constraints, 302—303 


with individuals differing in their ages and levels of expertise conducted in 
a variety of domains, 840 


of innovation, 301—302 


intended to induce changes in the rate at which expertise is acquired, 291 
of leader styles, 300 


of social innovation, 297—298 


experimentation, 446 


expert(s). See also human experts; older experts 
acquired skills through sustained practice, 23 


acquiring a large “vocabulary” or memory store of board patterns 
involving groups of pieces, 69 

acquiring knowledge by enrolling in courses, 6 

acquiring LTWM memory skills to encode relevant associations, 71 

acquiring specific mechanisms to adapt maximally to the constraints of 


their domains, 841 


acting within relevant constraints, 303 


activating suitable knowledge, 804 

adopting a particular image, 4 

advanced methods developed for eliciting and representing the knowledge 
of, 64 

advancing more accurate hypotheses, 333 

advantage over novices for random configurations, 69 


analyzing the taste components of wine, 238 


anticipating the evolution of the presented patterns, 683 

anticipation of, 362, 661 

arriving at a diagnostic decision without explicitly relying on biomedical 
knowledge, 802 


articulating how they believed a careful, rational person should reason, 92 


articulating the knowledge underneath their thinking, 92 
as an ascription from a social perspective, 127 
attaining their solutions in qualitatively different ways, 63 


attending to higher level, strategic or aesthetic issues, 540 


attracting attention from multiple observers, 304 

attributes acquired by during lengthy training, 11 

best qualified to evaluate their own performance, 4 

better at taking the perspective of others, anticipating the consequences of 
actions, and at improvising and adapting, 167 


better quality practice in their domain as well as across task domains, 260 


biomedical knowledge having implicitly encapsulated into procedures, 802 


caudate nucleus in synchrony with the infero-temporal and parietal areas, 
245 
during chess tasks showed a specific pattern of bilateral activation of 


homologous regions, 841 


in court, 141 


creating groups based on major physics principles, 69 
creating situations well suited on their own perceptual processes, 154 
criteria for, 67, 127 


defining standards, 143 


dependent on other people’s knowledge, 135 
described, 770 
developing a kind of “Long-term Working Memory,” 623 


developing better allocations of attention and sensitivity to critical cues, 
725 

developing skills to maintain rapid access to information, 16 

dictionary definitions, 3 


differing from novices in practice skill, 539 


direct access to the brains of, 233 


distinguishing in mathematics, 616 


doing things differently than novices at the level of cognitive processes, 
837 

driving more quickly through moderate waters, 360 

elaborated encoding of the current situation, 72 

eliciting knowledge from, 15, 91 


emotional reactions to cues, 464 


employing more fixations of shorter duration or fewer fixations of longer 
duration, 660 


encapsulated, domain-specific solution stored in long-term memory, 802 

encoding encountered information in LTWM, 71 

encoding representative information and storing it in long-term memory, 
70 

engaging in not only deliberate practice, but deliberate performance as 
well, 454 

engaging the head of the nucleus caudate, 243 

as “essential in domains where there are no right answers,” 143 

excelling mainly in their domain, 66 


as an exclusive group, 138-139 


exhibiting exceptional performance in only one area, 100 


exhibiting more flexible behaviors than novices, 359-360 


exhibiting predominantly breadth-first approaches, 384 


expecting signs to be on the right, 364 
extracting information from any time window of advance information, 679 
extracting knowledge of to build computer-based models, 12 


facets of the memory and knowledge of, 105 


failing gracefully, 73 


failing to make a contribution commensurate with their abilities, 434 

giving reports inconsistent with those of other experts, 192 

greater investment in the time committed to deliberate play by 12 years of 
age, 688 

having serious psychopathology, 224 

high level of knowledge, 807 


highest performance on the accuracy of diagnosis, 803 


highly experienced failing to demonstrate superior memory, 699 


in high-stakes, time-pressured decisions relying on experience and pattern- 
match, 457 


historical predecessors of modern, 127 


identifying, 746 


identifying a set of, 436 
identifying at an early age for sport, 666 


identifying in a particular domain, 36 


immediately examining solutions of high quality, 234 


incorporating knowledge in computer models, 12 
initial verbalized move later changed by a better alternative move, 74 


integrated both the visual and the technical elements of weaving, 385 


interviewing, 192 


invested more time in structured invasion practice, 687 

knowledgeable and skilled unable to simply transfer expertise to novices, 
33 

learning to perceive patterns that non-experts do not discern and to 
discriminate among stimuli, 454 

leaving behind a record of their work, 291 


leaving behind work plans, 304 


less likely to rely on formal cognitive processes than novices, 166 


likely to visit fewer cognitive states and to employ fewer cognitive 
processes, 180 
maintaining large amounts of information in working memory, 72 


managing time wisely, 456 
mechanisms affecting SA, 720—724 


mechanisms to circumvent processing limitations constraining normal 


performance, 836 


monitoring how they are thinking, 456 


more accurate in recognition of the patterns previously seen, 682 
more attuned to proximal sources of information in the action of their 


opponents, 680 


more likely to change behaviors, 360 


more likely to recognize the boundaries of what they know and don’t 
know, 100 


more varied than novices when the situation changes, 359 


move selection under speeded conditions not as good, on the average, 74 

needing to interact with people and the world within their areas of 
expertise and outside of it, 94 

not always exhibiting reliably superior performance, 14 


not vulnerable to base rate neglect, 335 


not well motivated to process information they already know, 805 


objective criteria for finding, 4 
in a particular domain, 34 


perceiving objects less well when inverted, 236 


perceptual, 151 
performance advantage for in test situations replicates the actual situation 


during competition, 206 


performance of, seeming impossible, 233 


performed worse on recall and explanations if judged by the simple count 
of propositions, 803 
performing diagnoses, 70 


pre-mission planning by, 724 


prestige obtained by, 45 
as proactive, 456 


processing visual images holistically, 159 


proposed models of the cognitive processes of, 696 


reasoning about observable features of the ECG, 204 


reasoning at levels more fundamental and defensible, 73 


reasoning process, 337 


recalled more information with short presentation times, 803 


recognizing familiar patterns, 69 
referring to someone who has learned from experience, 696 


referring to trained professionals with experience or credentials, 480 


reflecting on thought processes and methods, 72 
relying heavily on future projections, 716 
relying on the encoding of meaningful relations between laboratory data 


and “more extensive use of causal explanations,” 205 


“reproducible superior performance of, 616 


reproducing complex stimuli from game situations after only a brief 
exposure, 696 

reproducing more pieces in the presented chess position than a novice, 68 

reproducing the reliably superior performance of in a controlled setting, 13 

required knowledge already in long-term memory, 805 

running mental simulations, 455 


SA not severely constrained by working memory, 725 


seeing things that not-expert workers do not, 167 


selecting highly relevant features, processes, and esembles, 153 


selectively sensitive to task critical information, 464 


sensitive to changing conditions, 455 


showing greater or lesser fluctuations in performance, 229 


simulating motion, 247 

social roles of, 138-141 

as source of knowledge for expert systems, 85 

spending a greater proportion of time in initial problem evaluation, 63 
spending considerable effort at the task of situation assessment, 735 


storing and accessing information in larger cognitive chunks, 381 


strategies different from those of less skilled peers, 234 


strategies for dealing with the biases, 336 


struggled to differentiate between pairs displayed chronologically, 684 
studying recognized, 436 


superior hazard detection performance of, 734 


superior performance of, 11, 192 


superior reasoning ability, 205 
superior speed of reacting by, 16 


superior working memory, 696—709 


superiority found to be specific to aspects related to the particular domain 
of expertise, 10 
taking into account more ideas in planning a text, 420 


tending to anticipate more than novices, 362 


tending to have excellent long-term retention for domain-related material, 
341 


terminating regular engagement in deliberate practice, 753 


testing large groups of with psychometric tests, 10 


testing their own understanding and evaluating the value of partial 


solutions, 73 
thinking about problems in more abstract ways, 66 


thinking aloud while selecting moves, 200 


time-efficient use of knowledge, 141 


training techniques and performance limits of, 17 
trying to make lasting contributions in music, 540 


as unique, 291 


usability of knowledge, 71 
using clinical concepts with encapsulated biomedical knowledge, 803 
using more efficient problem-solving strategies than novices, 90 


using the information contained within structured patterns, 681 


viewed for centuries as mysterious, 696 


well attuned to affect in response to critical elements of the task context, 
464 


expert achievement 


in part based on past projects, 302 


requiring extended experience, 751 
expert advantage. See also expertise advantage 


consistent on anticipatory tasks, 679 


explanations of the evolution of, 680-681 


for information pick-up, 686 
over novices for random configurations, 69 


replicating the actual situation during competition, 206 


expert and elite performers, seeking out teachers, 76 
expert anticipatory skill, 680 
expert athletes. See also athletes 


accumulating significant hours in practice, 668 


better at using vision to pick up relevant information, 660 


fixating on areas of display that are more informative, 660 
as more accurate than less expert counterparts, 662 


perceptual-cognitive advantage using peripheral vision, 686 


expert authority, socially constructed nature of, 21 
expert calculators. See also calculators (human) 
exploiting long-term memory, 628 


learning numerical facts and procedures, 625 


manipulating numbers mentally, 240 
expert chess players. See also chess players; strong chess players; world-class 


chess players 
accessing the best move, 702 


discovering the more superior moves, 74 
evaluating chess positions, 702 

extracting chess relations in parallel, 599 
identifying the best move while thinking aloud, 697 


making fewer fixations per trial, 599 


making shorter duration fixations, 599 
not relying on transient short-term memory, 70 


recognizing and generating chess moves, 11 


relying on chess patterns stored in long-term memory structures, 599 


showing slower decline than less expert chess players, 847 
expert cognition, as the “goal state” for education, 64 


expert designers, 373. See also designers 


alternating rapidly in shifts of attention, 379 
appearing to be “ill-behaved” problem solvers, 386 
challenging problem “rules,” 376 


characteristic key strategies or approaches, 376—382 


having repertoires of “gambits,” 382 


not working from “intuition,” 386 


recognizing patterns in problem situations, 381 


tenacious in pursuit of solution concepts, 382 
using a mixture of breadth-first and depth-first approaches, 384 


working from first principles in innovative design, 383 
expert diagnosticians, organizing diagnostic hypotheses, 69 
expert drivers. See also drivers; experienced drivers 

retaining secondary task engagement across time, 366 


superior in both daytime and nighttime conditions, 362 


expert entrepreneurs. See also entrepreneur(s) 
achieving better calibration of asking, 402 


endogenizing causality, 401 
focusing on the downside to reduce the worst-case scenario, 396 


proactively “effecting” changes, 394 
working with means they already control, 395 


working with people who want to work with them, 396 
expert firefighters. See firefighters 
expert footballers. See footballers 


“expert in context,” as the minimum unit of analysis, 401 


expert knowledge 
about teaching, 34 
ameliorating difficulties of eliciting and translating, 92 


crucial for transportation, 358 
deployment of impacted by visual features, 153 
eliciting, 91—92 


having value to apps, 100 
of L2 vocabulary, 648 
not stored in pre-frontal regions, 838 


re-thinking, 34—35 

retrieved quickly using long-term working memory, 541 

stabilizing, 22 

translated into computer-readable representations, 87 
expert levels of performance 

attaining, 67 

defining, 272 

scientific study of, 756 


expert mechanisms. See also mechanisms 
as compensatory means for age-related decline, 843-844 


not easily learned or taught, 837 


sets of differing between younger and older experts, 839 


expert musicians. See also musicians 
accounting for differences among, 755 


competence and control beliefs, 542 


imaging the sounds of a piece, 756 


listening to their current performance, 756 
not showing superior memory for melodies, 699 


structural brain changes in, 239 


systems of representations, 756 
expert performance. See also expert performance approach; performance 
ability predictors for individual differences in, 223 


accounting for individual differences in the development of, 752—759 

affected by small variation in input variables, 43 

associated with automation and performance based on pattern recognition 
and direct access of actions, 12 


athletes improving performance characteristics, 276 


attaining as a sequence of states, 763—765 


body of evidence on, 207 

body of knowledge of documented, 77 
broadcast publicly, 44 

case studies, 302 

defined, 4, 480 


described by a distinctive age curve, 310 


deterioration in, 804 
developing, 35—37, 540 


differing based on type of technology, 366—367 
direct match to the progression toward, 756 


disrupted by inappropriate training procedures, 794 


drawing inferences about, 295—298 


enacted and embodied, 35 

features of, 44 

focusing on, 68 

generalizing across different domains of expertise, 10—14 
gradual increases in as a function of age, 750 
historiometric research and, 320—322 


implications for the study of structure and acquisition of, 709 


inseparable from individual and collective activities, 34 
interdependent with the performance of others, 221 
mechanisms underlying, 70 


multidimensional, dynamic, highly individualized and non-linear, 265 


observations of providing records, 291 


ongoing development of models of in other domains, 592 


predictors of, 223 


relying more on specific rather than general cognitive mechanisms, 840 


requiring deliberate practice, 43 


requiring integration of knowledge and skills, 35 
research determining how individuals achieve, 76 
shared psychological constraints and adaptations, 76 


as a Signal, 44 


as special and different, 708 

in sports defined, 272 

study becoming a science of learning, 77 
study of, 17 

superior anticipation and, 690 

tracing across time, 315 


underlying social innovations, 296 


using cases to understand, 291—305 


expert performance approach, 66, 745, 748 


effort to identify measures of performance, 751 


to expertise, 70, 699 


identifying cognitive activities tightly connected to the defining of expert 
performance, 699 

identifying mechanisms mediating expert performance, 68 

identifying representative tasks and reproducing the experts’ consistent 
superior performance, 72 

identifying task activities capturing the essence of superior performance, 
198 


precursor to, 198 


with protocol analysis and designed experiments, 200—203 


searching for reproducibly superior objective performance, 198 


searching for ways to study the effects of particular practice activities, 762 


to superior working memory, 699—700 


expert performance framework, 745 


expert performers 
acquiring associated working memory skills and LTWM mechanisms, 707 


attaining initial beginning levels of achievement, 18 


counteracting automaticity, 755 


improving performance for years and decades, 75 
magnitude of superiority exhibited by, 480 
as primary teachers at advanced levels (masters), 9 


recalling more information from structured patterns, 682 


regressing to non-expert performance levels, 682 
showing much smaller differences between them than novices, 220 
study of the development of, 77 
expert pianists. See also pianists 
age-effects reduced or fully absent in, 843 
depending on constancy in the environment, 433 


practice hours accumulated, 654 


professional activities in older compared with young experts, 845 


ranging in age from 50 to 70 years, 761 
reproducing exact timing and key pressures, 749 


younger spending more time on deliberate practice, 761 


expert pilots 
cognizant of emotions, 464 


having more anticipation in the direction of travel, 362 


relying more on long-term memory for SA than on working memory, 717, 
729 
expert reasoning. See also reasoning 
“base model” of, 184 


elements of, 176 


expert role(s), 127 


based on research from social psychology, 128 


redefinition of, 138 

rethinking, 138 

social division of labor and, 139-141 
“The expert” social form, 140, 141 


expert social performance, 298-300 


expert status 
in social groups, 23 
in social settings, 21 
expert superiority, involving “chunking,” 69 
expert systems, 12, 84. See also system(s) 
claims about expertise resulting from, 89 
combining the judgments of multiple with specialized knowledge, 99 
common goal, 62 
duplicating the performance of experts, 76 
emergence of, 89 
filling in reasonable defaults, 91 
foundational ideas, 84—86 
main areas for research on, 89 
as models of human expertise, 91 
perspective from computer science, 84—86 
relying on large amounts of knowledge, 92 
relying on symbolic knowledge, 85 
separation of key components, 84 
showing the rules connecting inferential steps linking primary facts, 94 
as a subclass of knowledge-based systems, 85 
tangible benefits of, 96 


variations in the implementation of, 98—99 


expert teachers. See also teachers 
consistent differences favoring designated, 437 
defined, 440 


in a dysfunctional school system, 434 


identifying, 437 

lacking a clear conception of, 431 
expert teams, 458-459. See also team(s) 

characteristics of, 509-510 


compared to a team of experts, 506 


contending with varying degrees of virtuality, 518 


current methods used to investigate, 518—519 


defining clear roles and responsibilities, 509 


fostering collective trust and a sense of team orientation and confidence, 
909 
fostering elements of, 522—52 


having strong leadership, 509 


managing and optimizing performance outcomes, 510 

members needing to respond adaptively to changing conditions, 459 

more than the sum of each individual team members’ level of experience, 
506 


number of publications related to performance of, 507 


as often “quiet” indicating “implicit” coordination, 519 


optimizing resources by learning and adapting, 509 


organization in alignment with the goal of cultivating, 522 

practicing a cycle of pre-briefing and debriefing, 509 

requiring a high degree of shared cognition in the form of both TMS and 
SMM, 513 


tending to discuss the problem and responses to it in great detail, 459 


working on ongoing rather than static “interactive team cognition,” 514 
expert technical performance, 300—301 
expert ways 
of being, 37 
as amyth, 76 
“The expert”-interaction, constituents of, 140 
expertise. See also expertises; human expertise 
achievement of, 64, 763 
acquired nature of, 76 
acquisition of. See expertise acquisition 
aging and, 835-850 


allowing retrieval from long-term memory, 420 


applications constructed to explicitly study, 60 
arising through specific adaptations, 665 


assessing from an ecological perspective, 776—777 


associated with other senses, 238 


assumed to constitute a continuum rather than a novice—expert dichotomy, 


794, 807 
assumption of separate independent components of, 36 


basis of, decoupled from general abilities, 843 


both context-sensitive and dependent on tacit knowledge, 24 
brain accommodating, 235 


bringing improvements in strategies, 358 


as a Capacity or property acquired by socialization, 21 
capturing and unpacking, 457-459 

capturing the essence of, 746 

categorizing along two dimensions, 93 


changing low-level perceptual experience, 157—160 


changing the functional and structural properties of the brain, 248 
in the classroom, 440—443 


cognitive mechanisms of, 233-235 


Cognitive Systems Engineering and, 167 


in commercial flying as a function of the aircraft, 358 
compensating for cognitive decline, 357 
in competitive activities, 17 


in complex institutions, 434 


conceived along a quantitative scale, 315 


conceptualizing, 771 


as a consequence of gradual accretion of knowledge, 336 


consisting of something other than knowledge and skills, 34 
continuous process of reaching, 277 
in creative thinking, 820-828, 832 


dependent upon embodied being in the world, 35 
depending on well-organized, specialized knowledge, 89 


developed and demonstrated in particular circumstances of practice, 111 


developing a broader and more complex understanding, 384 
developing attending to attuned responsiveness, 37 
developing in design, 384—385 


developing situational, 118 

development in the childhood work of Klee, Toulouse-Lautrec, or Picasso, 
592 

differences in the amount and structure of knowledge, 68 


as a different representation and organization of knowledge, 12—1 


as difficult to achieve and quite rare, 44 
as difficult to quantify and multi-dimensional, 95 
dimensions of, 27—28, 141-143 


discerning with indicators other than conscious intent, 49 


disordinal interaction with large differences in, 793 
displayed by non-humans, 15 
domains of, 3 


as domain-specific, 771 


in drawing, 589 
as dynamic, embodied, intersubjective and plural, 37 


ecological view of, 771-772 


effect on tasks involving surfaces drivers, 362 


effecting critical processes involved in developing SA in complex 
domains, 716 


effects of in a visual category, 158 


effortless mastery of as a myth, 76 

as elite achievement resulting from superior learning environments, 13 
enacted and embodied, 35, 37 

evolution of, 40—46 

in evolutionary and ecological context, 53 

examining the growth of as a function of time devoted to practice, 344 
as exceptional performance, 49-50 


exhibiting superior strategies for gathering information, 735 


extending to dynamic environments, 356 

as the extrapolation of everyday skill to extended experience, | 1—12 
factors of, 142 

functioning as a costly signal, 44 


gaining in complex domains requiring learning to see well, 578 


generalizable aspects of, 10—14 

generalizable characteristics of, 66 

generating sufficient revenue for a number of domains of, 9 
guiding the deployment of perception in complex tasks, 152 
hallmark in SA, 722 


held in and sustained by the activities of a social group, 23 


highest levels characterized by contextually based intuitive actions, 12 
identifying different levels of, 23 
ignoring the power of, 174 


including knowledge of how a system works, 466 


individual representations of varying, 112 


in interpersonal tasks, 230 


intertwining of knowledge and basic reasoning in, 67—68 
investigated using a seven country sample of entrepreneurs, 392 
involving deeper and more functional representations of tasks, 69—70 
involving extreme skill, 40 

involving larger and more integrated cognitive units, 68-69 
involving many more cognitive processes than just the simple act of 


perception, 237 


involving treatment of humans as much more difficult to measure, 746 


limited to a domain of knowledge, 66—67 
limited to a proscribed frame of reference, 93—94 


as a long-term developmental and adaptive process, 64 


long-term investment into, 849 

maintaining differing between athletes, 274 
in medical diagnosis, 95 

models of, 84 


modern day emerging from a confluence of factors, 40 


new methods for studying, 52—53 

new perspectives on, 4 

in non-human animals, 49—55 

not always necessary for high performance problem-solving, 93 
not leading to inflexibility, 74 

not necessarily scarce, 25 

objective definitions of, 49-51 

occurring in evolutionarily novel secondary domains, 46 
operationalising along three different dimensions, 27 


as an outcome of prolonged learning, 51 


overlapping variables of important to NDM investigations, 454 


in a particular domain, 719 


of particular social groups, 22 
partly defined by experts’ ability to explain their reasoning, 94—95 
perception in, 151-160 


as performance, 21—22 
postulated as playing a similar role in cognition, 820 
preservation and dissemination of, 96 


as a professionalized competence in differentiated domains, 128 


progressing through transient stages, 338 


as property, 23 
providing general benefits at advanced ages, 847-849 


psychometric approaches to, 213 


pursuit of, not much different from “normal” skill acquisition, 45 
putting demands on the brain, 239 

as real, 23-24 

reasoning and self-monitoring in, 73 


re-conceptualizing, 142 


relational or performative understanding of, 21 

as reliably superior (expert) performance on representative tasks, 13—14 
relying upon embodied being in the world, 35 

requiring dealing with the uncertainty of knowledge and assumptions, 91 
requiring practice over large periods of time, 436 

research on, 37 

resting on implicit assumptions about the context of a program, 95 


role in creative thinking, 812 


role in innovations of many sorts, 813 


role of perceptual routines in, 152—156 
routine processing versus controlled, 73—74 
SAs and tightly linked, 734 


as scarce and expensive, 96 


seeming to impact perceptual performance, 158 


serving as the foundation for creative achievement, 826 


from short-term to long-term working memory, 70—71 
as a social construction, 50—51 
social signaling and, 44-46 


socially defined nature of performance criteria for, 402 


as a Status, 21 
structure of and its acquisition, 3 
studies of from psychological perspectives, 59—77 


study of associated with several specific measurement problems, 216 


sustained through both purposeful and deliberate practice in a domain, 389 
tacit knowledge and practical intelligence as, 772—774 

in teaching, 431, 436-439, 440 

training for, 459-460 

trait predictors of, 221-225 


transferring, 823, 839 


as truly domain-specific, 89 


trying to apply to novel situations, 832 


using perceptual processes for novel purposes, 155 


in venture formation inferred from participants’ responses to the script- 
scenarios, 392 


in classical composers, 315 


empirical findings, 317—320 


examining from the perspective of situated and social cognition, 401 


individual differences in, 318 


involving gradual improvements in abilities, 839 


requiring around a decade of committed training and practice, 317 
through socialization, 21—28 

expertise advantage. See also expert advantage 
most evident in memory-based skills, 838 


projecting video-sequences of game situations, 706 
in transportation, 359 


expertise analysis of creativity, 820 


expertise and expert performance 
experience alone not sufficient for the development of, 75 
general books on the topics of, 66 
proposing a general theory of, 9 
toward generalizable characteristics of, 66—75 
tracing the development of knowledge of, 5—10 
expertise effects 
face-selective voxels in, 74 
obtaining, 359 


expertise management, NDM-based, 458 
expertise perspective 


in the case studies presented here and elsewhere, 830 


on creative thinking, 812 


expertise reversal 
different categories of reported within the context of cognitive load, 801 


in training sensorimotor skills, 803-804 


expertise reversal effect, 17 
caused by novice learners benefiting more from free-choice practice than 


deliberate practice, 807 


in cognitive load theory, 793, 805 
cognitive load theory as a framework for, 794—797 
defined, 793 


empirical evidence for, 797—802 


implication for the design of instruction in different domains, 806 


most evidence for collected in cross-sectional studies, 798 


expertise reversal studies 
examples of, 800 


most conducted in technical and academic domains, 802 


not many including very high-level experts, 807 


expertise space, three-dimensional, 27 
expertise status, explicitly recognized within the group, 140 
expertise studies 
general approaches to, 66 
historical development of, 59-66 
history of, 59 
incorporating ideas and concepts from situated and social cognition into, 
401 


expertise view 
assuming that changes in the world are slow, 820 
assuming that presentation of a problem results in retrieval of knowledge 


from memory, 813 


assuming that the creative imagination is structured, 813 
depending on near- or already-established associative connections, 828 


evidence for, 812 


expertise-driven general abilities account, 839, 848 


expertise-related abilities, tested for pianists, 843 


expertise-related changes, in the visual cortex, 157 


expertises. See also expertise 
contributory and interactional, 28 
located within small groups, 25 
periodic table of, 24-25 


in possession of every person in a society, 25 


expert-level perception, networks serving, 157 
expert-level performance, seeking to deliver, 85 
“expert—novice” approach, 66 
expert—novice difference research, 65 
expert—novice differences 
in ability to perceive relevant information, 63 
in advance information pick-up, 680 


in gaze behaviors, 661 


experts/exemplary journeymen, having a greater number of shorter fixations, 
361 

explaining one’s behavior, as a method to make education more effective, 
207 

explanation, 94, 446 


explanations and descriptions, by co-workers, 114 
explanatory variables, defined at any level, 280 


explicit and implicit knowledge, distinction between, 107 


explicit connections, making between problems and concepts, 439 


explicit consensus, for team members, 168 
explicit instructional guidance, providing to novices, 806 


explicit instructional support, 797 


explicit knowledge, 165, 168 


explicit monitoring, 804 
explicit processes 


accessible to conscious awareness, | 79 


in team coordination, 508 


explicit task-relevant knowledge, 803 
exploratory activities, yielding knowledge, 775 


explosive detection dogs, rating, 51 


expressive power, of computational methods, 100 
extended activities (running and climbing), capturing superior performance in 
the laboratory, 749 


“extended cognition,” 461 


extended engagement, in domain-related activities, 750 


extended narrative, composing, 415 
extended practice, 754 
extended professional experience, associated with decrements in 


performance, 753 


extensions, into more knowledge-intensive fields, 65 
extensive experience, in a domain, 745 


extensive reading, 638, 647 


extensor muscles, inhibition of, 567 


extemal contingencies (constraints), influencing expert performance, 302 
external feedback, for writers, 421 


external instruction, substituting for knowledge, 805 


external memory aids, removing access to, 703 


external memory device, sketching as, 380 
external representations, 379, 380 


external scene, coordinating with visualized values, 153 


external social judgments, based on ubiquitous knowledge, 25 
externalized plans, intended only for the writer’s private use, 414 
extraneous explanations, ruling out, 295 


extraordinary performance, expertise as the base of, 142 


extraordinary skills, relating to changes in brain anatomy, 559 


extrapolation, 446 


extreme base rates, problem of, 220 


extroversion, among experts, 224 


eye and hand movements, in naturalistic drawing, 591 
eye fixations 


during actual soccer games, 706 


patterns of, 72 


in situations when a hazard was signaled, 363 


eye movement (visual search) patterns, of experts, 679 


eye movements 


fixating the first (non-optimal) move that came to mind, 604 


information fixated during sequences of, 198 


relationship to anticipatory and interceptive skill, 684—686 
eye tracking methods, comparing experienced and novice teachers, 443 


eye—hand span, as the best predictor of typing speed, 706 


eye-line, half-way down the head, 584 


eyes, embedded in a face, 159 


eye-tracking 
results, 155 
studies, 645 
techniques, 598—599 
using with NDM, 465 


faces 
holistic processing of, 159 


images of produced in a pixel drawing task, 588 


importance of explicit domain-specific directly supported with regard to, 
984 
instant perception of, 235 


perception involving different aspects, 237 


positioning the eyes too far up the head, 584 


presenting the pinnacle of perceptual skill, 235 


separated into top and bottom halves, 159 
upright and inverted, 158 


facial geometry, brain region involved in processing, 156 


factor analysis, 279 

factor-analytic intelligence studies, 485 

factual knowledge, 84, 165 

“Faculty of Royal Designers for Industry,” 373 


faded worked examples, 806 
failures 


to craft knowledge for readers, 424 


feedback necessary for diagnosing, 399 
initiating learning from mistakes, 114 


resulting in new information becoming available, 832 


Fallingwater (house), Frank Lloyd Wright’s design of, 302, 825—826 


familiarity, 661 
families 
early instruction and support by, 13 
of high-achieving children, 536 
influencing the acquisition of extraordinary expertise, 317 


supported musical activities, 536 


family background 


as influential in the context of elites, 143 


world-class expertise emerging from, 317 


“far” object, seeming larger than the “near” object, 582 
“far” task, 664 


far transfer, denoting remote transfer, 821 


“fast,” concept of, 74 

fast finger movements, 552 

fastball sports, 660 

faster start, for outstanding composers, 319 
faults, 71 

“faute de mieux”-mechanism, peer review as, 137 
feedback 


about performance weaknesses and strengths, 480 


flow state requiring, 421 


on hazard perception as poor, 365 


for hazards usually vague and uncertain, 363 
immediate, 399 

on mistakes and problems, 754 

vs. no feedback, 800, 801 


on performance, 398, 779 


providing copious, 668 
providing knowledge of results, 424 


provision of rich meaningful, 464 


representing forms of partial instructional support, 800 


seeking, 459 
females, as more risk-averse than males in chess, 606 
Feng Wang, recalled 300 digits perfectly, 701 
FFA (fusiform face area), 236 
FGCs (fireground commanders), 456 


fiber tracts 


aligned in a less parallel manner, 560 


investigated in musicians, 560 


volume and direction, 559 
fiction writers, 425, 426 
fidelity, 341 
field hockey, 275, 655, 705 
“field notes,” 260 


field settings, exploring teams in a variety of, 518 


fifty-year olds, usually performing better than older athletes (sixty-year olds), 
274 

filament, 818 

film or video stimuli, permitting the vision of the opponent’s action to be 
occluded, 678 


films (movies), evaluated for each director, 321 


financial decision making, 97 
financial prospects, deciding among several, 479 


financial resources, starting a new venture on minimal, 396 


financial services industry, 97 


findings, comparability and generalizability of, 264—265 


fine motor control, 566 


movement production age-related declines and, 837 


fine motor differences, isolating as a function of artistic expertise, 591 


fine motor skills, developing in both hands, 550 
finger movement 

mental training of sequences, 558 

rapid, 566 


finger movement representation network, in the calculation process, 628 


finger or hand maps, intensive musical training associated with, 558 


finger placement, triggering the development of dystonia, 564 


finger sequences, young pianists and non-musicians performing simple and 
complex, 841 

finger tips, ability to hang with necessary for climbers, 749 

fingerprint matching, 236 

Finkelstein, Salo, 617, 622 


fire 


as also the solution, 819 
control of, 41 


creating a protective buffer zone, 830 


fire commanders 
expert decision processes, 465 


knowledge elicitation and knowledge representation undertaken with, 186 


firefighters 
expert, 71 
Klein studies of, 183 
wildland, 456 
fireground commanders (FGCs), 456 


firms, studying innovation in, 292 


first draft phase, of writing, 415 


first-person action video game playing, 591 


first-time entrepreneurs, 392 


fitness activities, 655 
Five Fs, 120 
“five stage model,” of Dreyfus and Dreyfus, 23 
fixation 
awareness of and attitudes towards by expert designers, 383 
effect in design, 382 
locations and duration of skilled and less-skilled soccer players, 664 
“flash anzan,” 626 
“flash of insight,” 379 
flashcards, 640 


flat associative hierarchies, remote associations and, 814-815 


flautists, the left hand more commonly problematic, 564 
Flexecution Model of (Re)planning, 461, 462-463 
flexibility 


of knowledge-based systems, 90 

retained at only the highest levels of expertise, 74 
flight hours, not significantly predictive of SA group, 729 
flight paths, of birds as spiral, 829 


flight scenarios, simulated with both inexperienced and experienced GA 
pilots, 728 


flight trajectory, processed in the parietal areas, 247 


flip of understanding, 819 
flow 


entry into facilitated by having a strong motivation to write, 421 


producing, 420 
fluency 


role in spoken and written form recognition and production, 642 


of vocabulary knowledge, 639 


fluent speech, producing under severe time pressure, 645 
fluid intelligence 

facets of, 838 

numeracy out-predicting, 493-494 


predicting general decision making skill, 491 


problem-solving dependent on, 607 


psychometric tradition considering facets of, 838 


“quantitative reasoning” tasks measuring, 485 


representing intelligence-as-process, 227 
testing for, 488, 493, 494 
fluid intelligence factor 


Carroll’s current, 494 


explaining overall general intelligence, 484 
fluid intelligence scores, historical increases in during the 20th century, 493 
fluid mental abilities, 836 
flying machine, Wright brothers’ invention of, 827-828 


fMRI. See functional magnetic resonance imaging (f{MRI) 
focal dystonia. See also dystonia 


abnormalities in three main areas, 566 


aetiology of as probably multifactorial, 566 
affecting high-level musicians, 544 


physical and psychological traumata, 566 


symptoms marking the beginning of, 563 


focusing on the big picture, experienced military officers rated higher on, 731 


folk biology, 42 
folk physics, 42 

chess building from, 43 
folk psychology, 42 


chess building from, 43 


followers, possessing expertise for mission execution, 299 


food manufacturing, 118 


football players, cannot be assigned to different teams, 221 
footballers, 656, 657 


forceps, requiring a high level of expertise, 444 


forearm rotation, degree of differing between pianists, 542 


forecasters, clear differences in proficiency among, 184 


forecasting 


future situation events and dynamics, 716 


impact on solving social innovation problems, 297 


studies focused on, 298 


foreign exchange trading, 97 
Forestalling, in RAWEFS, 457 
form, knowledge of, 641-642 
“form of life,” idea of, 23 
formal knowledge, 133, 780 


formal modeling, 610 


formal or informal settings, words appropriate for, 643 


formal rules, needing informal meta-rules, 24 
formal training, years since an individual initiated, 315 
formalization 

described, 133 


promoting professional standards and support technologies, 133 


requiring the cooperation of professions with other social institutions, 134 
formative assessment, skill at, 440 


formative years, participation characteristics during, 261 


form-based associations, characterizing a sparser lexicon, 643 


form—meaning links, 640 
formulaic language, 644 


aiding and speeding up language processing, 645 


as crucial for fluent language use, 644 
described, 644 
facilitating the production of fluent speech, 645 


as a hallmark of the highest stages of language mastery, 647 


importance of, 645 


use of by second language learners, 646 
formulaic sequences 

appropriate use of, 646 

defined, 644 

L2 learners improving, 646—647 


problem of choosing which to teach, 647 


processed faster in the brain, 645 


relying on the frequent use of, 644 


as standard ways of realizing functions and expressing ideas, 645 
as very frequent in language, 645 
ways of learning and teaching, 646 


forward chaining, 99 


Foucault, on the nature of governmentality, 131 


foundational methods, summary of, 181—182 
Foundations of Statistics (Savage), 479 


Four-Component Instructional Design (4C/ID) model, 806 


fractional anisotropy (FA), 559, 560, 561 


frame structure, 99 


framework. See also problem-solving model 
including knowledge of word parts, 642 


understanding and analyzing occupational expertise, 106 
framing 

effects in prognostic decisions, 335 

resistance to, 486, 487 


Franklin, Benjamin, 296 
fraud detection, 97, 98 
free recall tasks, 803 


frequency 
of measuring in a longitudinal study, 278 
with which hazards occur, 365 
of words, 637 
frequency distributions, words having different, 643 


fringe search, 301 
frontal-parietal circuitry, 838 
fronto-parietal network, 627 
Fry, Roger, 581 


full concentration, limits on daily durations of, 761 


Full Scale Numeracy, 490 
Fuller, Thomas, 624 


function words, 637 


functional assets, of experienced operators, 368 
functional brain changes 


in cognitive expertise, 259-244 


in perceptual expertise, 235—239 


functional brain imaging, 628 
functional heterogeneity, measured with a diversity index, 516 


functional leadership, 520 


functional magnetic resonance imaging (f{MRJ), 235 


comparing neural activities between musicians and non-musicians, 559 
highlighting localized brain regions mediating specific cognitive processes, 
415 


indicating perceptual regions associated with grouping, 155 


investigating the neurological bases of design cognition, 376 
study of poetry composition, 418 
functional nature, of experts’ task representations, 71 


functional networks, development of specialized, 156-157 


Functional or Ecological Interface Design, 170 
“functional task alignment,” 341 


functional use, 645 


functionality, within equipment, 174 


fundamental pitch, 543 
Fusiform Face Area (FFA) of the brain, 74, 156, 235, 236 
fusiform gyrus, 156, 561 


future, as an endogenous creation, 394 


future expert performers, acquiring initial and intermediate levels of 
performance, 77 


future experts, more efficiently training, 76 


future research, in entrepreneurship, 403—404 
future system states, projection of, 721 

future team research, opportunities for, 519-523 
future travel, 367—368 

Fuzzy-Trace Theory, 483 


g. See general intelligence (g) 

gabor patches, matching the orientation of, 619 
Gaelic footballers. See footballers 

Galileo Galilei, 135 

Galton, Francis, 193 


criteria of “eminence” in Hereditary Genius, 616 


eminence depending on “natural ability,” 618 


on high achievement, 835 


influential historiometric study, 311 
original proposal, 839 
social and educational effects of growing up in a talented and well- 


connected family, 626 


tripartite theory of eminence, 629 
“gambits,” repertoires of, 382 
gambling game, division of stakes in, 479 


game intelligence, 275 


game positions 
in chess, 601 
skill effect in the recall of, 600 


game sports, influenced by the performance of the opponent team, 273 


games 


analyzing between the very best chess players in the world, 758 


studies on age and expertise in, 840 


types of, 203 


Gamm, Rtidiger 
activations consistent with developing LTWM for arithmetical 


calculations, 628 


case study of, 617 


forward span of 11 digits and 12 digits backwards, 623 
investigation of working memory, 622 


solving multi-step problems quickly and accurately, 624 


teachers never explained arithmetic in ways he could understand, 625 


training up to four hours a day, 625 
gap, between mental content and expert performance, 37 
“garden-path” problem-solving, 74 


“gatekeeper role,” of disciplines in science, 137 


“gatekeepers,” in companies, 141 


gathering information and following procedures, experienced military 


officers rated higher on, 731 
Gauss, Carl Friederich, 616 
gaze, 120 
behavior, 661, 679 
directed to where the target is expected to be in the future, 685 


directed towards body segments, 684 


in driving, 360 
as flexible with expertise, 361 


less constrained with expertise unless there is a threat, 360—362 


measurement, 680 


more variable in the horizontal plane with experience, 361 
patterns of experienced and less-experienced vehicle operators, 360 
table of data, 664 

gaze-contingent displays, 686 


gaze-tracking systems, 685 


Gc. See crystallized intelligence (Gc) 


GE Plastics, configuring a color formula, 97 
gender, in chess, 607, 608 


gendering, of science, 137 


gene combinations, attainable performance of individuals and, 762 


general abilities. See also abilities 


correlated with performance for beginners, 67 


influence greater on performance of beginners, 708 
predictive of individual differences in performance, 746 
general ability factors, 70 


general and broad content ability measures, 223 


general aviation (GA) pilots 


analysis of SA problems in low-time, 727 


less experienced comparing to more experienced, 726 


as passive recipients of information (Level 1 SA), 726 


task prioritization and task management skills as important, 730 


typically far less experienced, 726 


general cognitive abilities, 490 


effects on expert decision makers, 497 


as weak correlates of expert performance in older age, 849 
General Decision Making Skill Assessment Battery, 490 
general decision making skill, 481-482. See also decision making 


acquired, 494 


best-fitting model of overall, 493 
conducting studies of, 489 
predictors of, 476, 482 


referring to differences in judgment and decision making quality, 477 


General Fluid Memory and Learning, 494 
general inference methods, 85, 89 


general intellectual capacity, 629 


general intelligence (g) 


decision making skill and, 483—485 


at early stages of skill acquisition, 836 


exact nature of yet unknown, 775 


links with life outcomes, 483 


as an umbrella term, 477 
viewed as a relatively stable characteristic, 775 


general pedagogical knowledge, 441 


General Problem Solver (GPS) computer program, 11, 60, 63 


general skill training, efficient, 496-497 


generalization, across diverse domains of expertise, 9 


generals, probability of winning a battle, 315 

“generate and test” method, 61 

“generative” reasoning, used by more experienced designers, 377 
generic-cognitive skills, primary knowledge including, 795 
genes 


influence of individual, 762 


located on the X-chromosome, 564. 
genetic endowments 


factors involving individual differences in, 762 


known immutable limits imposed by, 344 


relevance of, 317 


genetic influences, evidence for, 536 


genetic predisposition, 551 


genetic selection, influencing elements of physical and sensory capacities, 52 
Genetic Studies of Genius (Terman), 312 

genetic susceptibility, to malfunctioning of neuronal networks, 564 

genetics 


controlling for animals, 52 


as a factor in dog physical skill and expertise development, 52 
impact on mathematical computation, 626—627 


role in exceptional achievement, 312 


genetics researchers, supporting the idea of an innate domain-specific system, 
627 
genius and talent 


associated with distinctive education and training, 317 


insights into the origins of, 317 


not randomly distributed across space and time, 318 
geniuses 
designation of, 10 


sample of unquestionable, 312 


genomes, with most information borrowed from ancestors, 796 
geographic dispersion, of virtuality, 517 


geographical areas, operations in new, 728 


geologists, acquiring mental transformations, 153 


geometry component, of numeracy, 488 
Geometry Theorem Proving Program, 87 


geoscientists, recognizing word-forms fractured like rocks, 152 


German professional musicians, suffering from MD, 564 


Gestalt psychologists, concepts of restructuring and insight in problem- 
solving, 813 
“getting there” phase, 540 


Gf. See fluid intelligence 
glances, of apprentice drivers, 360 


global and local levels, shifting attention between, 586 


global context, characteristics of, 463 


global organization leaders, cognitive tasks, 463 


Global Positioning Systems (GPS), determining total movement within a 
practice, 258 
GM. See gray matter (GM) 
Go board game 
structural study on the experts of, 244 
variants of CHREST applied to, 600 
goal(s) 
already perceived, 717 


emerging through the effectual process, 396 
formation of, 119 


as ideal states, 717 


recursive structure with subgoals, 63 
teachers having clearly defined, 440 
understanding types of, 386 


goal state, representing, 61 


goal-directed action, accounts of, 114 


goal-directed nature, of teaching, 433 
goal-directed reasoning, as backward chaining, 99 


goal-directed training, intensity of as extremely high, 553 


goal-driven processing, alternating with data-driven processing, 717 
goaltenders, superhuman reactions of expert, 257 

golf, role of SA in, 714 

Gombrich, E. H., 584 


good decisions 


defined by logical processes, 479 


plausible real-world correlates of, 486 


good guessing, knowledge underlying the art of, 85 


good judgments, 440 
good performance 


involving not just the known “normal” situations, 723 


on one case as poor predictor of performance on another case, 332 


Google-style statistical learning, 92 

governmentality, formation of, 131 

GPS. See General Problem Solver (GPS) computer program; Global 
Positioning Systems (GPS) 


graded readers, use of, 647 


graduated driver’s licenses, 357 


grammatical functions, of a word, 640 


grammatical patterns, word classes following, 643 


grammatical usage in a text, counseling a user on, 98 
grandmasters 


able to reach international level years faster, 751 


able to reproduce entire chessboards, 1 1 
advantage in perception and memory, 579 
carrying out shallower searches than Masters, 602 


choosing better moves than the experts, 602 


reaching the status of, 67 

searching less than weaker masters, 605 

simulating playing against, 703 
granularity, in research on purposeful and deliberate practice, 398 
graph literacy, 494, 496 


graphemic representations, 414 


graphical co-option, of the conceptual, 155 


graphical representations, of numerical expressions of probability, 494 


graphs 


building useful mental models of, 496 
children learning to read, 153 
gravitational wave physicists, 26 
The Great Mental Calculators: The Psychology, Methods, and Lives of the 
Calculating Prodigies (Smith), 617 
“sreebles,” 158 
Greek philosophers, science started with, 134 


Greek sophists, called a “profession,” 134 

Greek tradition, as central to the scholastic tradition, 134 
Gretzky, Wayne, 714 

Griffiths, Arthur, 624 


group activities, involving several trainees at a time, 757 


group comparisons, producing a bias, 554 


group-based practice, 757—758 
group-level military conflict and strategy, simulating, 43 
groups 
expert performance of, 747—748 
expertise as the property of, 23 
studies correlating performance of and their performance on general ability 
tests, 746 
growth spurt, individual differences apparent, 276 


Guernica (painting), 826 


guidance 


forms of close or interpersonal, 120 
partial forms of, 799 


used intentionally to assist learning, 119 


guided discovery approach, 689 


guided learning, 120 


guiding principles, offering starting points for imposing order, 375 


“guiding themes,” of experienced architects, 377 


“guild of masters and students” (univeristas magistribus et pupillorum), 6 
guilds, 5, 6, 8 
guitarists, right hand more frequently involved, 564 


gustative stimuli, 238 


gustatory expertise, 237 


gymnastics, smaller stature beneficial for, 276 


Habitat for Humanity, 296 


habits, based on experience of the world, 35 


habitual actions, constituting forms of knowledge, 35 
habituated actions, basis for, 35 
hairdressers 

principles and practices used by, 110 


responding to exigencies of settings, 110 


shaping work activities, 112 


hairdressing salons, sequencing activities in, 118 


“Hammerklavier” sonata, by Beethoven, 544 
hand, requiring neurons in the primary motor area, 556 


hand and eye movements, bidirectional influences of, 591 


hand areas, of the cerebellum, 558 

hand dystonia, 567 

handball coaches, 274 

handball players, 706 

handbook, general outline of, 14—17 

Handheld Standoff Mine Detection System, 174 
handicrafts, 7 


“handover,” communication breakdowns during, 347 
handwriting, 423 
haptic skills, required for shaping clay, 117 


“hard modularity” approach, to cognition, 42 


“hard-to-learn” knowledge, premises relevant for constructing, 120 
Hawthorne Effect, 260 


hazard awareness, 734 


hazard detection, training for drivers, 364—365 


hazard perception. See also perception 


experience as a poor way of acquiring the skill, 365 


improving, 363, 365 


predicting crash risk and on-road driving performance, 362—363 


thought to be a non-automatic cognitive process with a substantial top- 
down component, 363 


training for, 364 


hazards 
missed more often by the apprentices, 363 
as relatively infrequent, 363 


head movements, during actual soccer games, 706 


headhunters, no studies of, 402 


head-mounted eye movement registration systems, recording visual point of 
gaze, 660 
health condition, impact of, 847 


health professionals, relations among, 129 
healthcare 


as an application area for Cognitive Systems Engineering, 187 


delivery as a primary concern, 331 


settings, 347 


simulations used for research and training in, 465 


healthcare professionals, having specific and narrow expertise, 188 


healthcare system, changes in, 340 


Healthcare Technical Group, within the Human Factors and Ergonomics 
Society, 187 

hearing losses, from overexposure to noise, 544 

“heat map” representation, of eye fixation areas, 604 

hedges, 417 

Heidegger, Martin, 34 

height, 752 


helicopter flight, fixations on the instrument panel, 361 


helpfulness experiment, 404 


help-seeking behavior, affected by perceptions of other’ willingness to help, 
402 
hereditary factors, role in dystonia, 565 


Hereditary Genius: An Inquiry into Its Laws and Consequences (Galton), 10, 
311 


heritability, inferred from twin studies, 762 


Heschl’s gyrus, 559 


heterogeneity 
in extraversion trait among team members, 515 
in methods, 265 


heuristic deliberation, 483 


heuristic knowledge, 85 
heuristic methods, 821 
used by Watson and Crick, 823 


heuristic model, 565 


heuristic processing, referring to a failure of selective attention, 159 


heuristic-based evaluation, 482 
heuristics 
bound together through “effectual” logic, 394 


centered on control, 401 


defining general-purpose, 87 
example of the use of, 821 


few verified experimentally, 333 


involving expertise, 821 


from LT were generalized into a model, 60 


seeking to exert control over the environment, 394 


viewed as “hard-wired” and universal, 334 


Heuristics and Biases approach, 454 


hexagonal model of interests, 226 
hidden profile, as a common experimental design, 140 


hierarchical leadership, compared with shared leadership, 515 


hierarchical linear model, 280, 321 


hierarchical linear modeling, 217, 316 
High Ability Studies, special issue of on “Expertise and Giftedness 
Research,” 11 


high achievement, attributing to innate, stable dispositions, 835 


high achieving countries, teaching methods varying markedly, 439 
high-achieving athletes, developing skills, 654 
high-creative individuals, produced more low-frequency word-association 


responses, 832 


high-knowledge individuals, greater recognition and recall memory for new 


material, 68 


ee) 


high-level cognitive processes, 662—66 


high-level expertise, hard to acquire, 25 


high-level knowledge and planning, in chess, 604 
high-level performance, experts consistently demonstrating, 36 


high offices and positions, attainment of, 314 


“high performing” group, in soccer, 658 


high prior knowledge learners, 799 
high prior knowledge students, 800 
high speed scanning task, 361 


high-performance judgments, aided by computerized tools, 142 


high-performance sport, funding involved in, 653 
high-performing anticipation group, 687 
high-performing group, in soccer, 658 


high-performing older adults, distinguishing from those showing pronounced 
decline, 844 


high versus low-creative individuals, 832 


higher prior knowledge students, 798, 800 
higher reasoning in humans, 61 
Hindu Vedic priests, cortical thickness in, 244 


hippocampal and parahippocampal areas, 238 


hippocampus, 244, 555 


historic figures, samples out of the norm, 322 
historical background, of research on expertise, 697 
historical development, of expertise studies, 59-66 


historical increases, in levels of performance, 544 


historical individuals, 310 


historical records, 322 
historiometric data, analyzing complex subject matter, 305 


historiometric emphasis, on the nomothetic, 311 


historiometric inquiries, into the role of genetics in exceptional achievement, 


312 


historiometric inquiry, of Quetelet, 311 
historiometric methods, 310—322 


historiometric research 
as correlational rather than experimental, 316 
depending on significant samples, 317 


empirical results of, 317-322 


historiometric studies, diversity of research designs, 315 


historiometrics 
defined, 310 
history of, 311-313 
methodological issues, 313-317 


relying on biographical and historical data, 322 


“Historiometry as an Exact Science” (Wood), 312 


history-making achievements, complexity of, 315 


hockey experts, anticipation skill in, 247 


holistic processing, 159, 235 
holistic ratings, 590 


holistic-like process, typical of radiological expertise, 236 


hominid brain size, 41 
homme moyen (“average person”), 311 


Homo sapiens, engaged in visual art, 576 


homogeneity 


in research on purposeful and deliberate practice, 398 
of teaching practices within countries, 434 
homophones, 642 


“homuncular” order, 556 


honors, receipt of major, 314 


hospital-based critical care, SA errors, 715 
hospital-based critical errors, 716 

hotel room attendants, patterns of progression, 118 
Hou Yifan, Chinese Grandmaster, 608 


hours of practice, necessary for musical instruments, 537 


hours of work, passing with minimal awareness of the passage of time, 420 
the “how,” of The Ask, 398 


“how to” knowledge, automatized with practice, 773 


how-to manuals, training artists throughout history, 584 


human(s) 


capacity for accumulating crystallized intelligence, 273 
competing against each other, 41 

computer expertise and, 65 

controlling a plane in the air, 827 

developed “expertise” in functional domains, 40 
exhibiting expertise in differentiated domains, 45 
forming broad multilevel networks, 40 

functioning effectively, 444 

machine systems and, 60 


by nature social and embodied, 402 


as predominantly visual creatures, 576 


using a tool, 772 


human capital, 391 


human cognition. See also cognition 
differentiating us from other species, 41 
interplay of intuition and deliberation, 480 
models of, 42 
Human Cognitive Abilities — A Survey of Factor Analytic Studies (Carroll), 


484 
human cognitive architecture, 795—797 


human development, experiential nature of, 772 


human domain experts, knowledge-based methods of, 62 
human expertise. See expertise 
human experts. See expert(s) 


human flight, Leonardo having a long-standing interest in, 829 


human genome, size of, 796 


human long-term memory, providing a functional equivalent of a genome, 
796 


human chess master, building, 606-610 


human mind, classical view of, 233 


Human Problem Solving (Newell and Simon), 11 
human—human and human—automation teams, 467 
human-machine systems, designing of, 65 
Husserl, Edmund, 34 

hybrid ecologies, 466 

“hybrid” entrepreneurs, starting new ventures, 390 


hybrid environments, 466 


hybrid model of learning, for entrepreneurship, 393 


“hybridized” professionalism, 132 
hydroelectric power, Wright obtained, 825 
hypotheses 


about mediating cognitive processes, 193 


generating rapidly and effortlessly, 332 


for mechanisms mediating superior performance, 198 


testing on large samples of historical individuals, 311 


hypothetical constructs, asserting to capture contributions to performance, 60 


hypothetical probes, 176 


hypothetical restructuring, of Carroll’s cognitive ability model, 494 
“hypothetico-deductive method,” 332 


hypothetico-deductive model of reasoning, 336 


iconic sources, progressing to generic types, 381 


ideas 
expressing, 645 
new arising in two ways, 822 


new most valuable as the result of remote associations, 815 


remotely associated, 812 


retrieved from long-term memory, 423 


identification criteria, for formulaic sequences, 644 


identified mechanisms, mediating increases in performance, 198 
identities, known and not interchangeable, 313 

IDEO industrial-design consulting firm, 824—825 

ideological leaders, 299, 300 

idioms, 644 


if-then rules 


chaining to form a line of reasoning, 98 

as inferential knowledge, 90 

specifying a precondition and an action, 85 
illegitimate events, 293 
illness scripts, 72, 120, 337 
illnesses or diseases, remembering particular, 120 
ill-structured domains, 393, 397 


ill-structuredness of problems, in the early stages of design, 384 


illumination, 814 
illusions, 582, 591 


illusory shared cognition, 514 


image manipulation software, 588 


imaginary scenarios, running, 41 
imagination 
drawings of objects produced from, 584 
of mathematical prodigies, 616 


imagined objects, interfering with perception, 157 


Imitation Game, 27 
immediate indicators, of learning, 440 


immediate memory, 697 


Immersive Simulated Learning Environments, 465 
immersive simulation platforms, 465 

IMOI model, 508 

Imperial College Surgical Assessment Device (ICSAD), 341 


implementation, successful teaching always including, 445 
implications, of the Ask, 400-403 
implicit approaches, to learning, 688, 689 


implicit knowledge, 165, 168 


implicit learning, 455 


implicit perceptual training, 689 


implicit processes, 178, 179, 508 


improvements 


of cognitive performance, 195 


identifying opportunities, 399 
observed in skilled decision making, 496 
plateauing, 393 
Improving Diagnosis in Medicine (Balogh, Miller, & Ball), 332 


incidental acquisition, of formulaic language, 647 


incidental learning, 639, 689 


incidental memory for music just played, 541 


inclination, of mathematical prodigies, 616 


inclined lines, 155 


inclusive communication, in expert teams, 459 

increased age, bringing increased job-specific knowledge and skills, 840 

increased time available for planning, increasing quality of the moves 
selected, 201 


incubation stage, of creativity, 814 


Indian musicians, showing interesting problem-solving strategies, 544 


indicators, associated with high standards of performance, 167 


indirect cue—hazard connections, 364 


indirect precursors, 363 
indirect scenarios, 364 
individual accomplishment, 27 


individual activities and sports, rated high for enjoyment, 655 


individual aptitude 
contributing to expertise, 347 


showing a consistent positive relationship to outcome, 345 


individual attainment, gauging, 314 


individual differences 


\<o) 


accounting for in the development of expert performance, 752—75 
in chess, 607 


variables predicting outcomes in performance, 45 


individual excellence, professional expertise and, 142 


individual level data, aggregating to the team level, 515 


individual level leadership, compared to shared leadership, 515 


individual performance, measuring in domains with treatment outcomes, 


ia 187 


individual states, affecting performance on testing occasions, 214 


individual traits, parsing the sphere of, 224 


individualized coaching and teaching of students, importance of, 806 


individualized practice, 656 


teacher or coach designing, 761 


without a coach, 758 


individualized training, 554, 755, 757. See also training 


individuals. See also persons 


acquiring memory skills, 701 


developing the expertise of, 444 
discriminating between high and low quality, 44 


engaging practice activities with full concentration, 759 


exhibiting reproducibly superior performance in mental calculation, 703 


expertise as the property of, 23 


having to meet to be professionally successful, 106 


highly experienced, 753 


improving performance of, 750 


learning occupational practice, 121 


looking for data to either confirm or deny their assessments, 718 


with more knowledge and experience, 70 
observing, measuring, and analyzing performance in teams, 747 


remaking occupational practices, 112 


superior levels of performance in specific domains, 836 


talented, success of leading them to practice more, 763 


training individually in darts, bowling, and chess, 758 


variation between, 796 


indoor climbing gyms, 749 


inductive approach, to identifying experts, 437 
inductive factors, 485 


“inductive inference,” 485 


inductive logic, 479 


industrial design 
case study, 824 
students, 385 
industrial designers, 383 


industrial production, 372 
industries, investing in knowledge preservation projects, 849 


ineffective moves, jettisoned, 796 


inefficiency, of novices, 720 


infants, capacities with numbers, 617 
inference engines (inference procedures), 84, 99 
inference methods, 85 

capturing expertise, 99 


inferences 


confirmed by studies using other methods, 295 


strength of about expert performance increasing, 304 


inferential (outcome based) case studies, 295 
inferior (PHG) brain area, 243 


inferior parietal lobe, 246 


inferred reconstructions, 198 


influence, as a signal, 519 
influential chunking theory, 597 
“informal” learning, 108 
information 

bottlenecks identified, 174 


connecting new with already existing, 239 
distributing across team members, 723 


from the external environment, 822 


from failure, 822 


generating action appropriate to the environment, 797 
obtaining from others as biologically primary, 796 


rapid categorization of, 718 


transmitting in complex ways, 41 
information change, understanding of relative priorities and frequency of, 717 


information flow, between cortical and sub-cortical areas, 556 


information gathering task, 301 


information loss, in artworks, 576 


information pick-up, 619, 774 
information processing 


applicable to operational contexts, 453 


characteristics of evolutionary theory, 795 


invariant limits on, 76 
models, 61, 598-606 
program of primitive, 61 


progressive modification of individuals, 116 


view, 72 
viewpoint, 61 
Information Processing Language (IPL), 59, 63 


information stores, 795, 796 


information technologies, 466 


information technology specialists, 170 


informational value, 517 


information-loss model, of age-related slowing, 838 


information-processing approach, in cognitive science, 64 
information-processing domains, with associated cognitive modules, 43 
inhibition 

abnormal demonstrated at the cortical level, 567 

lack of, 566 


weaker between global and local processes, 586 


initial design ideas, dominant influence of, 382 


“initial states,” transforming, 64 


initial success, acting as self-reinforcement, 537 


initial task performance, relationship with the amount of learning, 217 
innate ability, role of in human expertise development, 54 


innate basis, of numerical abilities, 617 


innate dispositions, 836 
innate domain-general talents, 839 


inner speech, 194, 195 


inner-directed control, form of, 132 


“innocent eye” 


arguing non-veridical percepts interfere with veridical perception, 581 


mixed evidence for, 582 


proposed by art historians John Ruskin and Roger Fry, 581 


strong version not supported by empirical evidence, 583 
innovation, 143, 301—302 


Innovative Applications of AI, annual conference, 96 


innovative solution, fixated on the underlying principle of, 383 
input, facilitating a process, 508 

input-output models, 183 

input—process—output (IPO) model 


describing constant change within a team, 510 


of team effectiveness, 508 


inquisitiveness, methods facilitating, 783 


insight 


versus analysis as modes of thinking, 817 


arising from restructuring, 817 
remote associations and, 819 


reported when the solution was the first word that came to mind, 831 


instantiation, of a problem in a domain, 86 

Institute of Medicine (IoM) of the National Academy of Science, report 
published by, 331—332 

institutes, 130 


“institutional imperatives,” guiding the work of scientists, 137 


institutionalization 
puzzle of, 143 


of science, 135-136 


institutionalized expertise, research on, 141 


institutions and authorization area, of sociological and anthropological work, 
22 
instruction 


guiding processing of unfamiliar information, 805 


nature of depending on the type of expertise being acquired, 318 
purpose of, 804 


supporting construction of knowledge, 805 


by teachers during childhood and adolescence, 755 


instructional assistance, research on optimizing, 797 


instructional behaviors, determining the frequency of, 260 


instructional design implications, 794 


instructional guidance (worked examples), 797 


instructional methods, 806 


instructional objects, 445 


instructional or training methods, selection of, 806 


instructional routines, 442 


instructional strategies, 338-339 


instructional support, 17, 806 


instructional techniques, 793, 804 


instrumental groups, subdividing into single instruments, 564 


instrumentalists, requiring perceptual and motor skills different from non- 
musicians, 543 


instruments, relative rarity of certain musical, 564 


instrument-specific pitch perception preferences, 543 


insula area, 238, 240 


insurance companies, assessing risk, 97 


intangibles, for creating a new venture, 398 


integral teamwork elements, facilitating, 522 
integrated understanding, 481 


integrative conception, of expertise, 770 


integrative optimization techniques, 479 
intellectual abilities, 836 


intellectual capacities, fixed, 484 
“intellectual component,” of professional work, 133 
intellectual trends, extrapolations about perceived, 484 


“intellectual/cultural” trait complex, 226, 229 
intelligence, 619. See also artificial intelligence (AD); crystallized intelligence 
(Gc); fluid intelligence; general intelligence (g); practical 


intelligence; psychometric intelligence 
denoting stable, individual differences in domain-general abilities, 835 


determined by genetic factors, 484 
due more to the methods than to the knowledge, 87 


predicted superior decision making in young adult participants, 486 


as a Strong determinant of decision making quality, 484 


Intelligence, special issue on “Acquiring Expertise: Ability, Practice, and 
Other Influences,” 11 


intelligence analysis, 172 


intelligence information, techniques for gathering, 730 


intelligence test 
originator of, 617 


performance stable from one occasion to the next, 221 


scores on correlating with a criterion of academic performance, 216 


intelligent behavior, understanding the mechanisms of, 84 
intelligent people, tending to acquire higher levels of decision making skill, 
488 


intensive practice, advancing the writing skills of college students, 425 


intensive writing, steady and reliable gains for students, 425 
intentional experiences, securing rich learning, 115 


intentionality, exercise of, 112 


interaction between processes, as dynamic, 663 


interactional expertise, 53 
acquiring, 21, 27 


defining via domain-specific language competence, 139 


depending on immersion in the relevant community, 25 
intellectual roots of, 26 
as a means to compete socially, 53 

interactions 


among planning, translating, and reviewing, 415 


effect on the dependent variable, 280 


exploring between perceptual-cognitive skills, 663-665 


tracking between performers, 258 


“interactive team cognition,” 514 


interactive tools, conceptualizing and encoding expertise, 92 


interconnected words, network of, 640 


-_= 


interdependence, of performance, 220—22 


interdisciplinary collaborations, having a complex division of labor, 26 
interdisciplinary contrasts, 317 
interests, hexagonal model of, 226 


interfering task, reducing recall only marginally, 600 


inter-individual differences 


central to psychological measurement and psychometrics, 213 


found in cognitive approaches of investigating expertise, 112 


in intra-individual change, 217 


inter-individual preconditions, 105 


inter-individual variability 
of hours of deliberate practice to become chess masters, 609 


during learning or skill acquisition, 217 


substantial changes in typically found only for tasks within the capabilities 
of nearly all learners, 217 
inter-judge agreement and reliability, 294 
intermediate effect 


in clinical case studies, 802 


demonstrating that encapsulated knowledge is an important feature of a 
medical expert, 803 

in studies of medical expertise as manifestation of the expertise reversal 
effect, 804 


intermediate footballers. See footballers 
intermediate levels of expertise, 805, 806 


intermediate social structures, research on, 141 


internal attention, spreading broadly, 816 


internal consistency, 263 
internal judgments, 25 


internal memory resources, tracking the current step, 155 


“internal model,” 557 
internal quality control, 137 
internal representation of numbers in LTM, 704 


internal representations, 757 


international competitions, winning, 67 
international contexts, preparing professionals for work in, 465 
International Journal of Sport Psychology, special issue on the “Nature 
versus Nurture in Sport,” 11 
international level 
in chess, 750 
training and engagement shown to be necessary to reach, 755 


international chess masters, 602, 605 


international medical graduates (IMG), performance on a written licensing 
examination (USMLE 2 — Clinical Knowledge), 345 
inter-observer reliability, 215 


interpersonal domain, predicting expertise in, 230 


interpersonal exchange, revealing prevailing condition—action linkages, 781 


interpersonal interactions, expertise in, 229 
interpersonal processes, pedagogic practices comprising, 117 


interpersonal skills, jobs highly dependent on, 230 


interpretation, of cases, 294—295 


interpreters, 704, 705 


inter-professional competition, based on a specific kind of knowledge, 133 


inter-professional cooperation, requiring “relational expertise” and “relational 


agency,” 141 


interruptions 
creating a significant challenge for SA, 724 
effect of, 335 


informing process, 335 


minimal impact on accuracy, 347 
with no effect on accuracy, 335 


inter-subjectivity, role of, 401 


intertwining, of knowledge and basic reasoning in expertise, 67—68 
interval capacity, longitudinal changes in, 281 

interval endurance capacity, 275, 281 

interval level rating scale, 597 

interval shuttle run test, 281, 285 


interval training, for long-distance running, 759 
interventions 


designing for developing older adults’ potentials, 849 


to overcome limits, 544 


interview analysis, 279 
interview studies, of designers, 373 
interviewing 
not as successful as interactive discussions, 92 
techniques comprising face-to-face discussions, 261 
“jn-the-loop,” 719 


intra-individual comparisons, making during youth, 285 


intra-individual differences 


referring to differences within individuals, 213 
relegating to measurement error, 228 


intra-individual fluctuations in performance, 229 


intra-individual preconditions, 105 


intrinsic motivation, in writing, 424 

introspection, 192, 194 

introversion, positively correlated with skill in male chess players, 607 
intuition, 92, 112 


intuitive judgments, interplay with deliberation, 74 
“intuitive” way of thinking, in design, 373 


invariant approach, used in sports questionnaires, 261 


invasion activities, transfer value of, 688 


invasion-game team sports, research completed in, 687 


inventors, not wishing to become entrepreneurs, 402 


inverse optics problem, 584 


inversion costs, 158 


inversion effect, 158, 236 
inverted fingerprints, experts struggling with, 236 


inverted stimuli, impaired the performance of skilled radiologists, 236 


investment theory, of Cattel, 836 


investments, small step, 396 

investor “pitch,” 399 

investors, obtaining resources from, 399 
Iowa Writer’s Workshop, 424 


iPhone, as a noteworthy social innovation, 296 


IPL (Information Processing Language), 59, 63 
IPS (intraparietal sulcus), predicting performances on fine tactile 


discrimination tasks, 238 


IQ 
differences between musically trained and untrained, 848 
not distinguishing the best among chess players, 10 


obtaining reliable estimates of score, 312 


providing a reliable and valid indicator of academic success or failure, 222 
tests providing predictive validity, 222 

Irving, John, 426 

“TS-A” or “PART-OF” hierarchy, 99 


isolated components, vs. interacting components, 800, 801 


isolated—interactive elements technique, 800 


Italian Renaissance, 135 


Italian sociology, notion of “elite” introduced within, 139 


items, recommending to Internet shoppers, 98 
iteration, of analysis, synthesis and evaluation processes, 378 


iterative process, with understanding driving the search for new data, 718 


iterative refinement, of a knowledge base using case presentations, 92 


jam sessions, improving performance, 538 


“Jamming,” 536 

Japan, mathematics teaching following a different cultural pattern, 434 
Japanese pottery workshop, apprentices progressing in, 118 

jazz 


communal practice, 538 


guitarists starting much later, 536 


with improvised structures, 566 


soloists requiring extensive practice in groups, 536 
job design, informing, 291 
job knowledge, 773, 774 
job types, classification of, 223 


JOHNNIAC computer, at the RAND Corporation, 60 


Joint Forces Air Component Commanders, training scenarios for, 458 


joint problem-solving, 120 


journal clubs, 425 


Journal des savants, 136 
Journal of Cognitive Engineering and Decision Making, special issues in, 
454 


journalists, learning by doing, 425 


journals, 136 


journeymen, 5, 361 


judges 
of case studies, 294 


providing accuracy ratings for drawings, 590 
judgment. See also decision making 
accuracy of experts, 142 


combining with feature inspection, 783 


compared to decisions, 476 
exercising responsible, 37 
of expert teachers, 442-443 


tasks with irreducible uncertainty, 777 
Judgment and Decision Making (JDM), 453 
juggling, compared to calculating abilities, 618 


Juicy Salif (lemon squeezer), 374 


junior and senior grades, of proficiency, 454 


“Jurisdiction,” 133 


Kanfer-Ackerman Air Traffic Controller task, 219 
karate, 248 


Kasparov, playing simultaneous chess games, 603 


Kepler, Johannes, 136 


— 


keyboard players, brain use and changes in, 560, 56 
kinaesthetic feedback, 555 


kinematic information, 680 


kinetic information, stored in the brain, 245 

kinetics, differentiating among experts and novices, 341 
“king defense configuration,” as a chunk, 69 

Klein, Wim, 624 

knot tightness and stability, measures of, 341 


Knowing Differences theme, 784 

Knowing Process theme, 784 

Knowing Self theme, 784 

knowing that (declarative knowledge), 455 

Knowing Why theme, 784 

knowledge. See also abstract knowledge; accumulated knowledge; acquired 
knowledge; canonical occupational knowledge; declarative 
knowledge; domain knowledge; domain-specific knowledge; 
elicited knowledge; expert knowledge; factual knowledge; formal 
knowledge; high prior knowledge learners; implicit knowledge; job 
knowledge; meta-level knowledge; musical knowledge; negative 
knowledge; occupational knowledge; procedural knowledge; 
shared knowledge structures; specialized knowledge; tacit 


knowledge; vocabulary knowledge; word knowledge 
about what is not known or should not be done, 107 


accessing and applying to improve students’ learning opportunities, 441 
active engagement and construction of, 121 


allowing experts to deal with more complex situations, 364 


benefits of overlapping and complementary, 459 


compilation and automatization of, 121 


components of, 89 


of the conditions under which disease emerges, 337 


as the “currency” of competition, 133 


delineating different modes of, 107 
delineating the meaning of, 587 
depth of in vocabulary, 639-640 
described, 165 


developing by accessing workplace activities and interactions, 114 


different types and qualities of, 107 


different usages of the concept, 107 


of a domain, 89 

of educational contexts, 441 

of educational ends, purposes, and values, 441 
eliciting the right, 168 


embedded within a larger conceptual framework, 93 
of experts, 5, 165-188 
experts’ usability of their, 71 


extensive, well-organized, 797 


formal and public, 93 

forming a line of reasoning, 99 

of formulaic sequences, 646 

groups with a high and low level of, 68 
informal and private difficult to elicit, 93 
integrating new, 86 

as a key component of chess expertise, 599 


of learners, 441 


linked to social values and interests, 132 


new co-production of, 138 


no longer seen as a “nuisance variable,” 67 
passive view of, 99 


perceptual organization principles and, 597 


power and primacy of, 481 


providing access to what might not otherwise be learnt, 118-11 


relation with performance, 345 


relevant to a decision, 98 
required for expert teachers, 440-441 


required for expert-level performance, 87 


requisite dependence on for expert performance, 771 


secondary, 795 


strategies to encourage mastery of relevant, 339 
studying the structure of, 70 


teaching focusing on acquisition and application of, 338 


transferring to a computer program from an expert, 91 
of which aspects of a system or events are relevant, 721 
of a word, 644 

knowledge and skills 
focus on acquiring, 33 
formed and organized into embodied ways of being, 36 
integrating into being expert, 35 
role of, 391 

Knowledge Audit (KA), 458 


knowledge base 


containing explicit representations of knowledge, 85 
continued maintenance of, 92—93 


of an expert system, 84 


modifying, 86 
over-simplified model of with an inference engine, 90 


preventing rapid, significant changes to, 797 


of radiologists, 234 


knowledge components, for a given word, 643 
knowledge crafting, 413, 423 


knowledge deficits, errors arising from, 334 


knowledge driven work, trajectory for, 180 


knowledge elicitation 


in Cognitive Systems Engineering, 166, 167—170 


combining with knowledge representation, 17 


as the core of the design process, 466 


for Decision-Centered Design, 166 

developments in the methodology and applications of, 187 
methods, 167, 521 

products of, 170 


protocols, 167 


revealing implicit knowledge by scaffolding workers, 168 


stopping point for, 168 


strategies, 168 
techniques making tacit knowledge explicit, 781 


uncovering elements of expert reasoning, 176 


knowledge encapsulation theory, 802 


“knowledge engineers,” with social skills, 91 


knowledge intensive domains, creation of models of, 178 
knowledge management initiative, 781 
knowledge models, 178 


knowledge representation, 89 


developments in the methodology and applications of, 187 


integrating with the process of knowledge elicitation, 178 


organizing elicited knowledge, 170 


of a program, 87 
Knowledge research category, for transportation, 357 
knowledge restructuring, 106 
“knowledge sharing,” 93 
knowledge structures 
different from other stages, 337 
information about, 228 
in long-term memory, 805 
other kinds of, 90 


proposed, 333 


knowledge telling, producing a string of assertions, 423 
knowledge “transfer,” problem of, 35, 37 


knowledge transforming, yielding complex argument structures, 423 


knowledge usability, problems associated with overload or inefficiency, 71 
knowledgeable learners, exploring and solving relatively new tasks, 798 
knowledge-based methods, applied to logic and mathematics, 88 
knowledge-based paradigm, in AI, 85 
knowledge-based system, properties of, 86 
knowledge-based systems. See also expert systems 

applications of, 95—99 


constrained in scope, 100 


focus on accuracy, 84 
in general, 85 
history of, 86-87 
knowledge-based theory, of reasoning, 336 


knowledge-driven influences, facilitating perception and drawing accuracy, 
584 
knowledge-laden task environments, AI addressing, 62 


knowledge-related generative processes, skills based on, 541 


knowledge-rich programs, experimenting with, 87 


L1 Hebrew learners of L2 English, 646 
L2 (second language) 
learners improving use of formulaic sequences, 646—647 
learners knowing how words behave in context when surrounded by other 
words, 643 


learning a new word as first and foremost a matter of relabeling, 642 


practical suggestions for gaining expertise, 647-648 


vocabulary, expertise in, 634—649 


labels and referents, giving thoughts verbal expression, 195 


laboratory drawing tasks, 578 
laboratory studies 


of hazard perception, 363 


presented in support of the remote-associates view, 831 


uncovered mechanisms mediating instances of exceptional memory 
performance, 701 


of the use of both temporal and spatial occlusion techniques, 679 


land transport, expertise in, 356 
landmine detection, 172—173 


Lang Lang, legendary pianist, 552 


Langley, Samuel, 827 

language. See also English language; L2 (second language) 
adapting to better suit the learning biases of the neural mechanisms, 157 
affecting “thought,” 152 


described, 644 
facilitating skill acquisition, 53 
producing through handwriting, 415 


role of in expertise development, 53 
language chunks, accessing quickly, 644 
language input, increasing through extensive reading, 647 


language of a specialism, expertise in, 139 


language processing, 414, 622 


language users, 51, 644 
laparoscopic (pin hole) surgery, 764 


laparoscopic cholecystectomy surgery, 344 


laparoscopic inguinal hernia repair surgery, 342 


laparoscopic surgeons, 343 
laparoscopy skills, 343 
“large meaningful patterns,” characterizing expertise, 68 


large team, having several observers with assigned responsibilities, 168 


“larger patterns,” of experts, 66 
last work, seldom the best, 321 


late Acheulean tools, required secondary competencies and self-control, 45 


late mature, 276 
latent growth modeling, 217, 279 
later adulthood 


deliberate practice and expertise maintenance in, 842—843 


neuropsychological substrates of expertise in, 841-842 
lateral areas (DMTG and SMG), 243 
lateral occipital complex (LOC), 236, 238 


lateral temporal areas, 241 


latex glove, musicians suffering from dystonia playing with, 568 


Latin, instruction in, 6 
law 
certification to practice in France and the USA, 9 


as a distinct social system, 137 


law professions, tied to justice, 132 
“lay” audience, expert addressed in front of a, 140 
laypeople, placing their trust in professional workers, 129 
leader(s) 

leaving a rich body of archival material, 298 


mistakes by historically notable, 295 


problems, successfully solving, 300 


requiring different educational experiences from creators, 318 


substituting rankings and rationale for those of experts, 460 


of teams, 509 


working to reduce conflict in highly virtual teams, 518 


leader extraversion and cognitive ability, as predictors of initial levels of 
leadership efficacy, 520 
leader performance, cases of, 298-300 


leader problem-solving, judges appraise cases of, 294 


leader styles, experimental studies of, 300 


leader types, 300 
leader-member exchange, 520 


leadership. See also shared leadership; team leadership 
directing and organizing team members’ interdependent actions, 508 


as a domain where expert performance is valued, 298 
effective, 298 

efficacy, 520 

expert, 778 


expertise, 778 


problems, 299 
styles, 299 


traditional forms of, 515 


virtuality-team performance relationship and, 517 
learned behaviors, cultivating an organizational climate supporting, 522 


learned domain, promoting good SA in, 724 


learned information, expert reviewing and modifying, 92 
learned mental models, 728 
learned “routine,” cues outside, 723 


learner characteristics (aptitudes), 801 


learner prior knowledge, as the single most important factor influencing 
learning, 806 

learner-manipulated variables, 799 

learners. See also novice learners 


acquiring formulaic sequences from reading, 647 


active, 114 

adopting different learning or performance strategies, 218 
advanced, 646, 806 

becoming expert performers after five or six hours of practice, 218 
effective, 121 


engaging in and mediating what they experience, 119 


feedback provided to during problem-solving, 800 
knowledge of, 441 


low knowledge studying worked examples, 798 


matching written subtitles to words spoken on screen, 648 


more experienced benefiting from methods, 806 


more experienced deteriorating, 807 


more experienced problem-solving with minimal guidance, 800 
more expert profited from continuous animations, 800 


more expert requiring additional resources, 805 


more knowledgeable focusing only on their weak areas, 807 


more knowledgeable needing to process redundant instructional guidance, 
805 


musical, 542 


needing to know a large number of lexical items, 639 


poorest performing having the most to gain, 217 
vocabulary size required, 637—639 
learning. See also deep learning; mastery learning; perceptual learning 
arising incrementally, 115 
in chess, 606 


distinguished from performance, 667 


effective, 64 


following a power function, 606 


guided by more expert partners, 120 


improving the efficiency of, 9 


inferred from changes in behavior over time, 667 


informed and shaped by the social and material work environment, 122 


mindful of opportunities for, 459 
monitoring and enhancing, 36 


not directly observable, 538 


not possible to predict actual, 116 

as a permanent change in behavior over time, 216 
to see well, 152 

styles, 776 

taking time, 440 


through discovery, 119 
through practice, 121 
through work, 120 


understanding in a particular setting, 36 
work activities and, 114-115 
to write in a specific domain, 419 


learning curves, 344 


learning environments, 13 
learning goals, formulating clear, 440 
learning mechanisms, accounting for the acquisition of everyday skills, 11 


learning objectives, 342 


learning opportunities 
construct of, 439-440 
creating in the classroom, 440 


producing student outcomes, 440 


teachers creating precise, 440 


learning processes 
conceptualizations of, 115-116 


different resulting in different representations, 541 


learning through work, 122 


learning-by-doing literature, in economics, 393 
learning-from-experience hypothesis, rejecting, 393 

lectures, teachers presenting the same material to all listeners, 757 
left DLPFC, 240 

left frontal brain regions, 416 

left hemisphere, 555 


left inferior frontal gyrus, 237 


left intraparietal sulcus, 627 


left parietal lobe, 627 
left premotor cortex, 240 


left primary auditory cortex, 554 


left superior parietal cortex, 415 


left superior parietal gyrus (SPG), 240 
left superior temporal gyrus, 559 


legal documents, incomprehensible to the general public, 424 


legal domain, 402 


legislation, recognizing the value of experience, 357 
legitimacy, 131 

legitimate events, in studies of leadership, 293 
Leibniz, G. W., role of, 134 


lemon squeezer, design process of, 374 


Lemonade principle, 395 
Leonardo da Vinci, 590 
aerial screw, 818, 829-830 


knew that air was compressible, 830 
leap of analogical thinking connecting two remotely associated ideas, 819 


routine system to understand any object, 578 


Leopoldina (early academy), 136 


less exceptional individuals, examining using case studies, 305 
less-experienced drivers. See also drivers 


problems with efficient intake of information in, 735 


visual information processing of, 359 
less knowledgeable students, 798 
“less obvious” recombinations, as “more creative,” 813 


less skilled players, gained most from planning for the easy problems, 202 


less talented, failure leading them to practice less, 763 


lesson(s) 
debriefing sessions following, 447 
functioning similarly to the designed practice tasks described by Ericsson, 
4A6 
separated in time, 433 
spending weeks or months on a single, 447 


success of depending on a teacher’s preparation and planning, 434 


lesson study 
Japanese practice of, 446 
as a lab for deliberate practice, 446-447 


providing teachers with skills to transform their daily work, 447 


less-skilled participants, using an experience-based model, 686 


less-skilled players, depending more on superficial elements, 684 
“letting go,” during a first draft, 419 
level(s) 

of achievement, 745 

of discomfort, 464 


of expertise, 794 


of specialization, 602 


Level-1 trauma cases, 510 
Level-1/2/3 verbalizations, 195 
leverage points, finding, 456 


lexical coverage, 635-637 


lexical decisions, tasks requiring, 838 


lexical space, metaphor of, 639 
“liberal arts,” in philosophy, 137 


life expectancy, controlling for, 317 


lifetime accumulated practice, related to the level of performance attained, 


537 
lifetime productivity, 314 
lifeworld, 34 
lifeworld perspective from phenomenology, 37 
light bulb, Edison’s invention of, 818, 829 
lightness constancy, 583 
Lilienthal, Otto, 827 
Lilies problem, 821 


limitations, of individual adaptation, 846 
limited attention, 717, 719 
limited working memory 

compensating for, 644 

effects on SA, 717 


limits, to human recall, 264 


linear functions, tracking, 153 


linear process, determining team performance, 508 


linear progression, 718 

linguistic abilities of humans, differentiating them from computers, 26 

linguistic discourse, 27, 28 

linked habits, theory of, 63 

lip representation, altered in patients suffering from embouchure dystonia, 
567 


lips, requiring neurons in the primary motor area, 556 


liquidity constraints, 390 

list structures, 60, 99 

listeners, recognizing individual spoken lexical forms, 635 
listening 


lexical coverage required, 636—637 


to music as a complex task, 554 
list-processing computer language, 59 
literacy 

foundations of established in toddlers, 422 


in the use of information and communication technologies, 37 
literacy skills, experts with, 127 


literature on NDM, bifurcating people into experts versus novices, 454 


“lived body,” overcoming problem of a gap between contents of the mind and 
expert performance, 35 
lived experiences 


of occupational being enacted, 117-118 


presenting in terms of cues, strategies, factors, and novice difficulties, 460 
LOC (lateral occipital complex), 236, 238 


local transfer, 821 


locations, along a route, 239 


logarithms, memorizing the table of, 624 
log/diary, 259 

logic, categories of, 479 

Logic Theorist (LT), 60, 63 

Logic Theory Machine, 61 


Logic Theory Program, proofs to theorems in Whitehead and Russell’s 
Principia Mathematica, 87 


“Logical Competitor Set,” for a case, 69 


“logical-mathematical” intelligence, 629 


logistic Item Response Theory models, 488 


longitudinal data 


absence of using comprehensive, multi-disciplinary test batteries, 666 


collected in a retrospective fashion, 263 


collecting over a period of time, 262 


two-level hierarchy defined in, 280 


longitudinal dataset, on practice-related variables, 259 


longitudinal designs, of historiometric studies, 315—316 


longitudinal measurements, permitting a direct comparison of performance 
improvement, 751 

longitudinal or prospective designs, addressing issues of predictive ability, 
266 


longitudinal research, nature of, 262 


longitudinal results, regarding shared leadership in teams, 520 


longitudinal studies 


acquisition of drawing skill versus various perceptual abilities, 589 


describing detailed development of the structure of expert performance, 
709 
detecting developments or changes in the characteristics of the target 


population, 271 


extending beyond a single moment in time, 271 


highly relevant from repeated observations, 271 


majority utilizing an aggregated design, 316 


measuring change in outcomes at the individual level, 285 


multidisciplinary, 271—285 
pointing to general gains in cognition, 848 


showing that a minimum of ten years of experience seems to be necessary 


to become an expert at an international level in chess, 751 
Statistical analyses in, 278-280 
studying the acquisition and maintenance of expertise, 271 


training novices to memorize chess positions, 606 


longitudinal study design 


applying, 275 


measuring soccer performance and technical skills, 278 


not as flexible as cross-sectional studies, 285 


long-lived creators, remaining productive, 320 


long-term commitment, predicting involvement and practice, 537 
long-term episodic memory, experts and, 622 
long-term goal, 276, 277 


long-term memory (LTM) 
accumulated knowledge of experts stored in, 696 
areas involved in, 244 
automatic retrieval from, 71 


differences in, underpinning skill differences, 605 


encoding of sequences of 3-digit groups, 199 
evidence for storage in with retrieval cues, 704 


exploiting the unlimited capacity of, 622 


guiding processing of familiar information, 805 


instances of radiological images in, 234 
large capacity of, 71 

number of chunks held in, 600 

rapid access to, 419-420 


size of having no adequate measure, 796 


stored, biologically secondary information obtained from other people, 796 


structure and contents of, 599 


structures circumventing the limitations of working memory, 718 
utilizing pre-existing knowledge in, 483 
long-term musical training, protecting the auditory system from age-related 
decline, 842 


long-term outcomes, effects on, 432 


long-term predictability, of talent tests, 836 
long-term talent identification programme, predictive utility of, 666 
long-term working memory (LTWM), 17, 622 


deployed by experts, 623 


encoding chess position in, 70 
model, 701 


“looked but failed to see” error, 723 


lookup task, with pairs of nouns, 218 


“losing,” oneself in work, 420 
Love Canal, 22 

low-coherency environment, 175 
low-fidelity models, 341 


low-fidelity simulation format, assessing domain-specific tacit knowledge, 
776 


low knowledge learners, studying worked examples, 798 


“low-level” engineering efforts, mathematically modeling neuronal cell 
assemblies, 63 


low-level perceptual processes, touched by expertise, 160 


“low-level” transfer, possible between sports, 669 


low-level visual processing, systematic changes in as a result of video game 
playing, 591 
low-physical-fidelity simulations, used by NDM practitioners, 460 
low prior knowledge students 
achieved better by studying worked examples, 798 
benefited from the added visuals, 800 


benefited from worked examples, 800 
low-validity environments, people performing significantly more poorly than 
algorithms in, 175 


LTM. See long-term memory (LTM) 
LTWM. See long-term working memory (LTWM) 


M1 Abrams tank turbine engine, 96 
machines, making decisions, 100 
macrocognition 


assuming a dynamic, ongoing and iterative engagement, 183 


combining concepts in with concepts from conflict management, 462 
extending the focus of NDM, 460-463 


macrocognitive functions, generally performed as collaborations, 461 


macrocognitive model, of expert forecaster reasoning, 185 


Macrocognitive Modeling, 183-184 


macrocognitive strategies, for problem detection, 466 


macro-social view, on expertise, 401 


macrostructure, of search in chess, 602—603 


magnetic resonance imaging (MRI) 
high-resolution, 559 
scanners, 238, 246 
magnetoencephalography (MEG) studies, 558 
Mailer, Norman, 425, 426 


maintenance 


of reputation, 295 
through deliberate practice, 840, 849 


maintenance practice 
for older experts, 837, 845 
role of in sports, 842 


making music, relying on voluntary skilled movements, 555 


maladaptive plastic changes 


aiming to reverse, 568 


in the brain’s sensory and motor networks, 566 


malformed collocations, 646 
mammal species, skilling up their offspring, 53 


“man in the head,” no specification of the processes of, 194 


management 
by discovery, 462 


performance on a daily basis, 446 


of resources, 358 


of time by teachers, 433 


“management by objectives,” 462 


management decision making, in medicine, 347 


managerial literature, 130 


managers, compared to entrepreneurs, 396 
managing 


memory, 625 


uncertainty, ambiguity and time pressure, 456 


Mangiamele, Vito, 625 
manipulables, early experiences of calculators involving, 622 
Mann Gulch fire, 830 


manual motor practice, of the non-dominant left hand, 560 


manufactured objects, configuration of, 97 

manufacturing, 97 

map experts, not showing better memory for traditional types of maps, 699 
MAPP computer program, 600, 602 


marathon- or half-marathon runners, effects of regular training, 842 
market, for entrepreneurship exercise, 406—407 


market research, realistic, 405 


market research studies, experts skeptical about, 394 


marketing campaigns 


examining performance in developing, 297 


formulating for a new product, 303 


marketing slogan, discourse of professionalism in, 130 


marriage counselors, 402 
Marshall, George C., 294, 298, 299 


martial arts group, negative age-effects, 845 


Marxist interpretations, focused on medicine and law, 129 


mass educational provisions, meeting the needs of industrialized economies, 
113 
master and apprentice, relation between, 9 
master checker players, interviewing, 87 
master chess players 
choosing better moves, 600 


finding good moves with minimal search, 604 


making good decisions under time pressure, 603 


memorizing at least 100,000 opening moves, 598 


new estimates of the vocabulary of, 602 


reproducing the chess pieces in a position on the board, 683 


masterpieces, 5 
masters 
organized existing knowledge, 6 


paintings, of sharing a profound fidelity to the visible world, 577 


Masters Driver’s License in the United States, 359 


mastery learning, 342 
mastery-based training approaches, 342 
mastiff dog breeds, 51 


match, between expertise and the problem, 822, 823 


math operations test, predicting wages, 485 
Mathematica software package, 88 
mathematical abilities, twin study showing concordance rates for 


monozygotic and dizygotic pairs, 627 


mathematical calculation 
deliberate practice in, 75 
development of expertise in, 16 


mathematical constant m, memorized 60,000 decimals of, 200 


mathematical expertise, 616—630 
brain systems for, 627—629 
describing, 618 


no evidence for differences in innate specific capacities leading to, 630 


mathematical ideas, connections with core underlying, 435 
mathematical models 


accounting for the fine structure of careers, 322 


characterizing humans’ apprehension of environmental conditions, 777 


mathematical precocity, of mathematical prodigies, 616 
mathematical problems, calculation of, 703 


mathematical skills, role in decision making, 477 


mathematical tasks, not studied, 628 


mathematically gifted children and adolescents, investigated, 628 


mathematically related proficiencies, evident in adults’ lives, 477 


mathematicians 


professional, 628 

tending to live less long than other scientists, 317 
mathematics 

based on detailed physical and control models, 93 


expertise in, likely to show a Matthew effect, 218 


often overly precise and awkward, 100 


producing an “instrumental” understanding of, 434 
“relational” understanding of, 435 
Mathematics Knowledge for Teaching (MKT), 441 
mating and survival, evolutionary imperatives of, 44 
Matthew effect, 218 


maturation, 276 


maturation and skill development, allowing for working animals, 54 
mature reasoners, 155 
maximal performance 
becoming a rigidly determinate quantity, 10 
in real settings, 847 
Mazumdar-Shaw, Kiran, 395 
McClelland’s categorization, of professionalization “from within” and “from 
above,” 131 


mean performance, in the early sessions of practice, 218 


meaning 
knowledge of, 642—643 
of a word, 640 


meaningful associations, encoding arbitrary information by, 700 


Meaningful Learning, theory of, 176 
means 


compared to each other, 280 


under control of entrepreneurs, 395 
Means, James, 828 
“means—ends analysis,” 61 
measurement(s) 


of change over time, 216—220 


of domain-specific reproducibly superior performance, 746 
multiple taken in a longitudinal study, 271 


of reliable individual differences among experts’ and professionals 


performances, 748 


measurement artifacts, 216 
measurement occasions, 217 
Measures of Effective Teaching (MET) project, 438 


“measures of musical talents,” 535 


mechanical ability, 749 


mechanical arts, 6 


mechanical engineering design, 380 


mechanical engineering designers, 378, 382 


mechanisms. See also acquired mechanisms; causal mechanisms; expert 


mechanisms; mediating mechanisms; perceptual mechanisms 
acquisition of, 708, 763 


allowing transformation from novice to expert in a domain, 725 


artists outperforming non-artists using, 579 


creating for team cooperation and coordination, 510 
executing expert performance, 76 

experts using knowledge of, 90 

for forging linkages, 820 

gradually acquired versus innately given, 745 
identifying, 699 


mediating, 199 


mediating chess expertise, 201—203 


mediating expert chess players’ superior ability, 203 


mediating memory performance for numbers, 700—701 


mediating performance on representative tasks, 763 


mediating superior performance, 749 
mediating verifiable expertise, 480 


necessary for developing good SA, 716—718 


underlying expert performance, 70 
underlying good SA, 724 
media, task requirements of different, 592 
medial (RSC) brain area, 243 


medial frontal cortex, 240 


medial frontal gyrus, 240 


medial superior parietal gyrus, 240 


medial temporal lobe, 244 
mediating mechanisms 


inferred from verbal reports, 199 


step-by-step development of, 199 


mediating processes and structures, producing superior performances, 64 
mediation 

described by phenomenology, 132 

experts’ intuitive actions and, 12 
mediator 


accounting for variability in team performance, 508 


performance as, 402 
medical and surgical expertise, predictors of, 344—346 
medical diagnosis, one of the first knowledge areas for expert systems, 96 
medical doctors. See also physicians 

developing expert levels of performance, 397 

experienced, more accurate in diagnoses, 699 

handwriting of, 4 


interactions with patients, 204 


learning to practice medicine effectively, 118 


mortality and morbidity meetings, 119 
relying on a higher-level representation, 708, 803 
roles of, 111 


societal expectations for, 109 


troubleshooting automobile engine problems, 89 
medical education, expert—novice studies, 65 
medical educators, providing instructions for visually diagnosing clinical 
cases, 783 
medical error, morbidity and mortality as a result of, 332 


medical expertise, intermediate effect in the encapsulation theory of, 802 


medical experts 
able to explain diagnoses, 73 
not always outperforming less experienced individuals, 802 


medical field, expertise reversal in, 802—803 


medical images, requiring perceptual diagnosis, 204 


medical practitioners, taking patient perspective, 27 
medical profession 

analyses of, 129 

tied to the value of health, 132 


medical records, managing for the UK National Health Service, 171 


medical schools, 336 
medical science, expert dogs or other animals useful in diagnostic science, 54 


medical situation, stressfulness of a simulated, 347 


medical students 
less experienced outperforming relatively more experienced medical 


practitioners, 802 


piecemeal representation of a diagnosis, 73 


reproducing the information in a case with all detailed biomedical 
knowledge, 803 


wanting to know more about underlying reasons, 100 


medical teaching, identifying the need for articulation of tacit knowledge, 785 


medical tests and treatment effectiveness, predictive power of, 495 
medicine 
Al addressing, 62 


career in, resulting in documented achievement, 344 


certification to practice in France and the USA, 9 

diversity of skills involved in, 344 

effective treatments associated with sets of symptoms, 91 

fundamental goal to treat patients to increase health and wellbeing, 
203-206 

future issues, 346-347 


language of modern as a formal system, 133 


predictors of expertise in, 345 


standardized testing influencing admission to residency programs, 780 


studied by thinking aloud procedure, 63 

study accompanied by performance on many achievement and aptitude 
tests, 345 

study of expertise in, 331 


understanding the development of expertise in, 332 


writing reports for colleagues, 136 


medicine and surgery, expertise in, 331—347 


meditation, altering consciousness in the service of writing, 422 
Mednick, on creative thinking, 814 
meerkats, 53 


members 


of communities of practice, 780 


of teams, expertise of, 747 
memories 

for facts, 624 

involving the hippocampus, 554 


managing and strengthening, 623 
memorizers, 239, 240, 244 
memory. See also long-term memory (LTM); short-term memory (STM); 


superior memory; transactive memory; working memory 
“accuracy” and “rapidity” of distinguished from “association,” 620 


circumventing capacity limitations, 498 


constraints of, 333 


declining with age, 225 

deliberately assisted by creating an internal map of a musical piece, 539 
domain-specificity in, 623 

exceptional, 623, 700 


of mathematical experts, 620—624 
for move sequences using blindfold chess, 605 


of music demonstrated by savants, 541 


playing a crucial role in motor expertise, 245 


remembering responses to a survey or test, 214 
skilled as a by-product of chess skill, 703 
memory competitions, 700 


memory demands, integral to selecting chess moves, 203 
memory development, role of knowledge on, 606 
memory encodings, of experts in chess, 708 


memory expertise, 841 


memory experts 


making better use of brain regions, 841 


superior performance on specific memory tasks, 700 


memory performance 


improved on the digit span task, 698 


increased with practice, 755 
memory processes 

accuracy of, 837 

in chess, 599-602 


memory recall, for positions in chess, 600 


memory research, on the encoding and retrieval processes, 199 


memory skills 
based on LTM, 702 


experts maintaining efficient access to diverse information, 13 


memory span 


increasing performance with training, 698 


testing in flute players at different levels of music skill, 707 


memory strategies, spontaneous use of, 68 
memory tasks 


capturing superior performance on, 700 


studying performance on, 11 
“memory type,” used by prodigies, 617 
memory-based applications, processes superseding, 343 


men, outperforming women in chess, 607 


mental abacus 
activating motor regions of the brain, 628 


carrying out another task in parallel not involving numbers, 626 


development requiring practice on mental addition problems, 704 


expanding by one additional digit estimated to take about 1 year of 


deliberate effort, 704 
more experienced calculators using, 626 


mental activity, detailed descriptions of, 166 


mental and material, dissolving the traditional conceptual split between, 35 
mental calculation, working memory during, 703-704 
mental calculators, 239. See also calculators (human) 
Inaudi, 700 
relied on the traditional multiplication table for single digit combinations, 
703 


mental capacities, individual differences in, 10-11 


mental content, “leaping out” into expert performance, 33 


mental models, 132 


advantages of, 721 

building, 460 

combined with modular plasticity, 41 
defined, 720 

embodying stored long-term knowledge, 721 


expertise highly dependent on, 720 
of experts, 455 


experts’ superior, 17 
generating, 41 
inaccurate, 514 


insufficiently developed, 728 


of judges influence appraisals of events and actions taken, 294 
overreliance on, 728 
providing a significant short cut for determining Levels 2 and 3 SA, 722 


providing cognitive mechanisms, 718 


providing “default” values to, 721 


similarity and accuracy jointly predicting task performance, 459 
stretching beyond limits, 724 
mental multiplication 


of numbers, 195 


task involving, 197 


mental representation(s) 


of an abacus, 704 


constructing what the text says, 416 


detecting minor, but consequential, deviations, 74 
having multiple functions, 73 


for implementing a plan on an instrument, 541 


mediating expertise during execution, evaluation, and learning, 72—73 
for musicians, 541 
students needing help to acquire, 76 


mental resources, 363, 722 


mental simulation 
alriving at assessments, 722 
envisioning a sequence of events, 455 
mental structures and their functioning, addressing, 61 
mental tests, developed by Galton, 10 
mental training, 558 
mentors, identifying, 464 


mentors and role models, focusing on the influence of domain-specific, 315 


Merleau-Ponty, Maurice, 34 
Merton, Robert K., 137 
meso-level structures, as coordinating mechanisms in temporary groups, 510 


message, expressing, 645 


meta-analyses 


on performance attributable to practice, 762 


on shared leadership, 515 


meta-analytic methodology, revisiting prior studies of personality, 390 


meta-analytic reviews, documenting age-related declines, 837 
metacognition, 398, 456, 498 


metacognitive control, of experts, 464 


metacognitive effects, exerting a direct effect on affective responses, 496 


metacognitive heuristics, correcting or circumventing costly mistakes via, 
483 
metacognitive processes, 462 
metacognitive skills 
honing, 459 


musicians regulating their practice themselves, 539 


relevant to developing expert performance, 277 

metacognitive understanding, of one’s abilities, resources, and constraints, 
486 

Meta-DENDRAL, discovered new rules of mass spectrometry, 88 

meta-expertises, as expert domains, 25 

meta-knowledge, MYCIN indicating the order in which to pursue different 
goals, 90 

metal detector, detecting landmines, 172 


metal sensing device, development of a new type of, 173 


meta-level knowledge, 90, 100 


metaplasticity, referring to sensitive periods in the lifetime, 566 


meta-processing, varying with experience, 365—366 
meta-rules, directing the selection of rules to be invoked, 99 


meta-strategies, experts having good, 724 


meteorologists, 153 


method of loci, 239, 240, 24 
Method-of-Loci strategy, 841 
methodological artifacts, 316-317 


& 


methodological issues, in any case study, 293 
methodology, 195, 277 
methods 


from Cognitive Systems Engineering, 175-181 


for measuring expert teams, 518 


resulting in different representations, 541 
MetLife, 97 


metrics, based on total practice time, 258 


Mexican birth attendants, skill acquisition of, 119 
Michelangelo, 576 
Mickelson, Phil, 714 


“microcognitive” functions, 461 


micro-genetic development (moment-by-moment learning), 112 


micro-level ethnographic studies, of professional socialization in workplaces, 
130 


micro-pitches, sensitivity to, 552 


microscopic changes, as the brain purges unnecessary connections, 245 


microscopic slides, of stained tissue, 204 
microstructure 


of deliberate practice, 656 


of practice, 258 


microsurgery proficiency and training, 342 
Middle Ages, craftsmen formed guilds, 5 
middle putamen, 557 


middle-distance runners, practice activities for, 759 


mid-frequency vocabulary, 638 


MIDI-technology, assessing finger movements, 560 


mid-life, animals reaching peak level performance in, 53 
Migration Period — the Dark Centuries, 134 
Miler, George, 61 


military command and control simulations, examining the impact of 


technology on team operations, 465 


military decision making, studies of, 461 


military experts, strategies to detect landmines, 173 


military officers 


experienced, concentrating more on the enemy situation, 731 


experienced, rated higher on SA related behaviors, 731 


learning how to gather and disseminate key information on the radio, 723 
preparing for operational decision making, 465 
military pilots. See also pilots 


experienced, engaging in practices or skills beneficial to SA, 726 


information available to dependent on specific actions, 723 
societal expectations for, 109 
tenfold difference in SA, 725 


mimicry, as a signal, 519 


minaret builders, 118 
mind 

comprised of computationally distinct mechanisms, 42 

shifting focus from to bodily perceptions, 35 
minicomputers, facilitating the manufacture of semi-custom, 97 
mini-lessons, planning to incorporate instructional routines, 442 
minimum unit of analysis, for expert performance involving other people, 

401 


Minotauromachy (painting), 826 
“mirror neuron network,” 558 


mirror neuron system, 681 


mirror neurons, 246 


misconceptions, explaining emergence of, 447 


misperception hypothesis, of drawing errors, 581 


missing information, ignoring, 91 

“mission critical” applications, 97 

Mitchell, Ronald, 392 

mixed designs, of historiometric studies, 316 


mixed methods, combining quantitative and qualitative, 207 


“mixed practice” strategy, 339 
MKT (Mathematics Knowledge for Teaching), 441 


mnemonic encoding, testing the role of, 199 


mnemonics 


reminding doctors about a series of interrelated conditions, 120 


used by artisans in Early Imperial China, 116 
mobile devices, widespread use of, 266 


modality of vision, 576 


“mode 2,” 138 
“mode-1/2 knowledge,” 107 
model(s) 


of expertise, 84, 90 
integrating macrocognitive functions and processes into NDM 
frameworks, 461 


for knowledge elicitation, 168 


providing as the designer’s goal, 380 


model essays, in the worked example condition, 798 


modeling projects, building functioning computer programs, 194 
moderators, identifying the impact of, 520 
modes 


of communication, 520 


of seeing, 154 
of transport, 356 
modular functions, supporting universal, primary competencies, 42 


molar-equivalence-molecular-decomposition approach, 844 


molecular modeling, to construct the structure of DNA, 823 
Mondeux, Henri, 623 


monitoring, 277 


monocular depth cues, 581 


mono-disciplinary, cross-sectional studies, 285 


monopoly of competence, 129 
monopoly practice, legitimacy of, 129 
Moorsel, Leontien Zijlaard-Van, 277 


moral community, professionalism as a form of, 128 


moral questions, feedback and, 498 


morally relevant biases, 498 


more experienced learners 
performance deteriorating, 807 
problem-solving with minimal guidance, 800 


more experienced students, benefited most from step-only guidance, 799 


more expert learners 
profiting more from continuous animations, 799 
requiring additional resources, 805 

more knowledgeable learners 


improved more by focusing only on their weak areas, 807 


needing to process redundant instructional guidance, 805 
“more like this” button, 98 


more versus less-creative people, differing in response hierarchies, 831 


morphology, of the corpus callosum, 561 


most frequent words, providing the greatest amount of coverage, 637 
“mother” disciplines, dominated research in entrepreneurship, 391 


mothers, experience with learning a musical instrument, 539 


motion 
of the arm holding the racquet in badminton, 679 
extraction of information on, 684 


motion pictures, evaluations of, 314 


motivation 
effect of, 281 
to engage in a task, 424 


indirect effect by moderating practice in chess, 609 


integrated into the L2 speech community, 646 


to maintain intense training, 847 
motivational component, to self-efficacy, 225 


motivational factors, individual-difference variables and, 592 


motivational rituals, practicing, 420 


motivations and intentions, considering a multiplicity of, 403 
motor actions, 556 


motor and visuo-spatial pathways, of children with abacus experience, 244 


motor anticipation, in fast-moving sports, 246 
motor associative areas, 240 
motor basal ganglia loop, 561 


motor brain function, 558 


motor component, compared to the visual component, 246 


motor control 
increasing degree of loss of, 565 
theories of, 342 


motor co-representations, activation of, 558 


motor domains, 233, 245 


motor excitability thresholds, reduced, 558 


motor expertise 


crucial neural component of, 246 


functional brain changes in, 246—247 


(pre)frontal areas in, 248 


structural brain changes in, 248 


motor familiarity hypothesis, 681 


motor homunculus, represented upside-down, 556 


motor mechanics, | 11 


motor performance, 749 


motor performers, expert, 681 


motor processes, 591 
motor proficiency (doing), 342 


motor skills, clinical procedures using, 340 


motor system, 555, 566 


motorcyclists, high fatality rate, 360 
move, deciding on the next in chess, 243 


movement, as not thought about movement, 35 


movement efficiency, differentiating experts and novices, 341 


movement pattern information, anticipatory encoding of, 681-684 


movement pattern of others, use of advance information from, 678 


movement perception and production, common coding view of, 681 


movement proficiency, 341 


movement responses, 678 


movements patterns, in response to visual cues, 556 


movie directors, 321 

Mozart, W. A., 540, 625 

“Mozartians,” 418 

MRI. See magnetic resonance imaging (MRI) 
Miiller-Lyer illusion, 583 

multidigit arithmetic, by calculators, 623 


multidimensional and longitudinal research 
increase in over time, 278 


overview of studies on, 282 


providing valuable insight in measures of performance, 285 


studies applying, 281 
multidimensional nature of expertise, considering, 276 


multidimensional performance characteristics, hierarchy of, 281 


multidisciplinary approach, existing NDM methodologies and theories 
taking, 467 
multidisciplinary longitudinal studies, from the field of sports, 271—285 


multidisciplinary talent identification model, 667 


multifaceted approach, using strengths from various techniques, 266 


multilevel analyses, revealing male and female elite youth players, 281 


multilevel data, regression analysis for, 280 
multilevel designs, latest advances in, 316 
multilevel modelling, 279, 280-281 
multilevel modelling program MIwiN, 281 


multi-method team performance measurement system, 519 


multinational organizations, using virtual teams, 516 


“multi-ness,” characterizing the current world, 467 


multiple choice licensing examination, as a better predictor of complaints to a 
regulatory body, 345 


“multiple players,” operational settings involving, 458 


multiple regression, appropriate for analyzing hierarchically structured data, 
280 


multiple regression analysis, 316 


“multiple-hurdles” approach, 220, 223 


multiplication, number of operations growing with the square of the number 
of digits, 621 

multiplication tables, 624 

multisensory connections between auditory and motor areas, piano training 
and, 558 

multisensory integration, tasks involving, 555 

multitasking. See also attention 

skills for, 730 


multi-team environments, leadership in, 509 


multi-team systems, 467 
multivariate voxel pattern analysis (MVPA), 236 


muscular-skeletal or neurological problems, of musicians, 544 


music 
affected by societal factors, 544 
brain regions involved in performing, 554—557 


cognitive adaptations in, 541—542 


experiencing, 554 


expertise in, 535—545 
as a grammar-based but non-semantic temporal phenomenon, 545 


individualized instruction in, 755 


international level success in composing, 751 


performance happening rarely, 445 


performing at a professional level, 551, 552 


performing in public, 553 


playing by sight (sight reading), 707 


playing depending on a strong coupling of perception and action, 550 
task specificity limitation to a domain, 67 
music aptitude tests, 535 


music composers. See classical music composers; composers 
music educators, confronted with assumptions of parents, 535 


music experts. See expert musicians 


music expression, aspired compared to actual music expression, 756 


music making, as a prime candidate for the study of complex skills, 545 


music memory, of savants, 541 
music perception, 554 
music performance. See also performance 


aesthetic quality of, 570 


anxiety, 553 
deliberate practice in, 75 
domain of, 552 


effects on brain plasticity, 553 


evaluation of by independent judges, 747 
music practice, start of, 756 


music students 


accepting jobs as professional musicians, 761 
beginners not correcting errors, 760 


overcoming initial lack of motivation, 538 


practicing by, 537 


musical achievement, individual differences in, 535 


musical activities 


early commencement of, 561 


inducing brain plasticity, 568 


linked to conditions of high arousal and positive emotions, 553 


musical and compositional preparation, indicators of, 319 
musical constraints, severe in classical music, 566 
musical experience 


advantages in advanced age, 848 


length of correlated with performance, 552 
musical expertise 

brain changes associated with, 550-570 

development of, 539-545 

modeling, 561 


predispositions and experience contributing to, 552 


research conducted in the classical conservatoire tradition, 536 


musical genres, 536 
musical instruments 


adaptation to the specific demands of different, 561 


growth curves associated with learning, 344 


musical knowledge, 541 


musical notation, 550, 555 
musical performance, error processing during, 556 


musical pitch information, transformation of, 555 


musical prodigies, information on, 536 


musical savants, special attention to, 535 


musical stimuli, processing of, 562 


musical structure, cognitive representation of, 541 


musical talent, individual differences in, 319 


musical tasks, 559 
musical training. See also training 


benefits of broader functions like IQ or school performance, 848 


cognitive and personality variables, 552 


curricula, 755 
effects on the brain, 543, 557—55 


of professional musicians, 553 


sustained leading to improved cognitive functioning, 848 


musically trained individuals 
advantage in signal-processing, 842 


compared with controls, 841 


enhanced brainstem representations of musical sound waveforms, 557 


structural differences between older and novices, 842 
musically trained middle-aged and older adults, 842 
musicians. See also expert musicians; professional musicians 

advantage in near-transfer for pitch, but not for words, 848 

brain plasticity of, 568-570 


common patterns in families, 536 


consistently winning competitions, 67 


continued activities of, 553 


demands imposed on changing over time, 543 


enjoying improvement but disliking actual practice, 538 


extraordinarily skilled exerting more effort and concentration during 
practice, 551 


frequency and loudness discrimination, 543 
getting the “big picture” of the piece, 539 


imaging parts of pieces, 756 


listening to tones of instruments, 543 


male, more likely to have musicians’ cramp, 565 


mental representations with associated retrieval structures, 541 


performing a piece of music with limited or no preparation, 707 


performing as a member of an orchestra, 747 


performing in-phase symmetrical finger exercises on a keyboard, 568 
physiological adaptations of, 542—543 
practice, belittling, 537 


recalling musical material better than non-musicians, 541 


recalling tunes after a single hearing, 623 
structural brain differences in young and middle-aged, 841 


studies of young and older, 840 


suffering from dystonia, 569 


training-induced brain changes transferring to non-musical cognitive 
abilities, 848 


musicians and non-musicians, engaging the SMA, 556 


musician’s dystonia, 550. See also dystonia 


loss of motor control in skilled movements, 552 


as a syndrome of maladaptive plasticity, 563—566 


music-score reading, 562 


mutual learning, 138, 403 
MYCIN program, 94 


concept of degree of sickness, 95 


diagnostic strategy, 90 

expertise measured through a series of evaluations, 95 
performance of, 95 

procedure for diagnosis transferred to other domains, 89 


using for tutoring, 100 


myelin cells, 553 


myelination, 244, 560 


“nAch” (need for Achievement), 224, 390, 391 
Nadia (drawing savant), 579 


naming network, in the brain, 627 


narrow limits of change principle, 796 


narrow reading, 647 


narrow-focus condition, for finding digits, 816 
Nash, John Forbes, Jr., 224 


national and international level athletes, activities for, 759 


national and local standards, in teaching, 433 


National Assessment of Adult Literacy, 477 

National Association of Stock Dealers, 97 

National Board for Professional Teaching Standards (NBPTS), certifying 
teachers, 437 


National Health Service Care Records Service, 171 


National Institutes of Health, training to write grant applications to, 425 


national licensing examinations, 345 


native speakers, 634, 644 
natural abilities, 311, 535 


“natural” decision making settings, versus artificial laboratory, 461 


natural expertise, 15 


“natural flying machines,” observing birds, 828 


natural information processing systems, described by five basic principles, 
795-797 
natural language speaking, 23 


“natural” limits, of performance existing for experts, 543 


natural performance environment, requiring complex perceptual judgments, 
678 


natural scenes, vertexes created by intersecting boundaries in, 157 


natural threats, newly emerged, 139 


natural work settings, interest in, 186 


naturalistic contexts, decisions in, 479 
Naturalistic Decision Making (NDM), 16, 132 
background and history of, 453-454 


complementary to the microcognitive approach, 461 


concept of expertise in, 454—456 


cross-cultural research, 465 


emerging challenges and future research, 463-465 


evaluation of expertise in, 454 
fields of, 386 
frameworks, 456—457 


insights and innovations regarding training and education based on 


expertise, 65 

as a leading framework for the study of expert decision-making in 
operational environments, 453 

macrocognition extending the focus of, 460-463 

methods, 465 


new designation for the paradigm of, 183 


’ 


as a promising approach to understanding cultural factors in experts 
sense-making, 465 

researchers, 453, 454 

technology and, 465 


training relying on low-fidelity approaches, 460 
naturalistic inquiry 


into cognitive work, 183 


within dynamic, high-intensity work settings, 166 


naturalistic intelligence, 774 
naturalistic settings, featuring words in natural speech, 642 
naturalization area, of sociological and anthropological work, 22 


nature 


drawing using cylinder, sphere, and cone, 578 


experimenting with in a controlled manner, 135 


nature theorists, expertise research hotly discussed by, 536 


nature vs. nurture, issue of, 312, 551 


nature vs. nurture debate, 52, 654 

NBA (National Basketball Association), centers close to seven feet tall, 43 
NBPTS Certified teachers, 437 

near task, skilled players fixating on, 664 


near transfer. See also transfer 
all creative thinking based on, 823 


critical in Dodge’s insight based on expertise, 830 


of expertise, 821 


role in Picasso’s conceiving the overall structure of Guernica, 826 


used by Wright brothers in developing their control system, 828 
Wrights’ expertise at work through, 826 


near versus far analogies, effects on problem-solving, 831 


near-associates perspective, 815 
need for Achievement (“nAch”), 224, 390, 391 


negative age effects 


if tasks require more complex processing, 837 
in memorization of musical materials, 845 
negative age-related changes, emerging as early as the late 20s, 838 


negative feelings, accompanying the early stages of writing, 422 


negative knowledge, 107, 115 


negotiation, 399, 403 
Nelson-Denny reading test of verbal ability, 488 
neo-behaviorism, 63 


neonatal intensive care nurses, 186 


nephrologists, diagnosing patients, 205 


nervous system, demands placed on by music performance, 550 


nested structure, of measurements, 280 


network disorder, musician’s dystonia as, 568 


“network of enterprises,” use of by Edison, 296 


network or relational model, of expertise, 22 


networked expertise, 106, 141 


networks, adapting to handle visual categories, 160 


neural adaptation, direct correlations with years of practicing, 842 
neural basis 


for abacus use, 626 


of face perception, 235 


of monkey numerical abilities, 617 
of simple retrieval, 627 


neural cell assemblies, 568 


neural changes 


in expert memorizers, 239 


more extensive in experts, 248 


in perceptual expertise domains, 233 


neural complexes, mapping, 160 
neural connections and circuitry of the brain, shaped by training, 754 
neural correlates, behind experts’ performance, 233 


neural differences, between two kinds of calculators, 239 


neural implementation, of expertise, 233, 235 


neural mechanisms 
for experts’ anticipation skills, 246 


underpinning expert anticipation, 690 


neural net simulation work, 839 


neural networks, building, 93 
neural regions, 152, 156 


neural responses, in musicians, 557 


neural scaffolding processes, 844 


neural structure, 561 
neural substrates 


of expert and “novice” mechanisms, 836-837 


selective recruitment of by expert readers, 157 


neural systems, development of, 156—157 
neuro-active hormones, supporting neuroplastic adaptations, 553 


neuroimaging data, 418 


neuroimaging methods, 415 


neuroimaging studies 
on blind people involving areas of the brain reserved for visual processing, 
238 


featuring actual movements, 246 


featuring exceptional memorizers, 239 
on gustative expertise, 238 


neuroimaging techniques, availability for research, 235 


neurological damage, leading to spatial “acalculia,” 623 
neurological functioning, undergirding all mental activity, 775 


neurological patients, arithmetical abilities selectively affected, 624 


neurological seizures, diagnosed from brief video tapes, 204 


neurological substances, of general intelligence, 775 


neuronal networks, 553, 569 


neuroplastic changes, upper limit for attainable performance, 544 


neuroplasticity, integral to adult learning and development, 754 


neuropsychological studies, of differences in musicians’ and novices’ brains 
in young and middle-aged adults, 848 
neuropsychological substrates of expertise, in later adulthood, 841-842 


neuroscience 


music training and, 542 


study of design, 376 
new information 


connecting with already existing, 239 


stimulating new ideas, 822—823 


“new look” movement, Bruner’s classic, 151 
“A New Name for a New Science” (Woods), 312 


new ventures, 395, 397 


New York State Regents Examination in Plane Geometry, 87 
Newell, Allen, 59, 61, 88 

new-genre leadership, 515 

Newton, Isaac, 134, 224 


“next best move,” selection of, 200 


Next Generation Air Transportation Systems (NextGen), 366 
Nightline television program, 824 

Nijinsky, Vaslav, 224 

No Left (Right) Turn sign, 364 

no pain no gain, 439 

Nobel laureates, 322 


Nobel Prize winners, analyses of biographical data on, 13 


nomothetic hypotheses, 310-311 


non-ability characteristics, relating to learning and performance, 775 
non-adaptive mutations, jettisoned, 796 


non-artists, baseline performance of, 583 


non-coach-led, sport-specific play, 659 


non-coach-led practice activities, 659 
non-compositionality, degree of, 644 


non-creative individuals, possessing a steep associative hierarchy, 814 


non-drivers. See also drivers 


controlling a powerboat simulator, 359 
driving with, 359 


with freedom to scan, 359 


of powerboats, 359 


slowed their boat going from slight to moderate seas, 360 


non-European music genres, studies in, 544 


non-expert askers, 402 

non-experts 
engaged the left Broca and the neighboring ventral prefrontal cortex, 240 
examining using case studies, 305 


superior decision making associated with a domain-general skill, 476 


using the right prefronto parietal network, 241 


non-functional domains, developing expertise in, 40 
non-functional expertise, 15 


non-laboratory performance, 838-839 


non-language using animals, evidence of instruction and skill scaffolding 
amongst, 53 
non-linguistic signals, during face-to-face interactions, 519 


non-monotonic, single-peaked function, with a mid-career optimum, 311 


non-musicians, controlling complex sequences, 841 


non-reactive verbal reports of thinking, 195-197 


non-representational style, of painting, 827 


non-routine events, stories about, 175 


non-routine problem-solving, engaging in, 112 


non-situation specific context, 665 

non-technical difficulties, interfering with success, 92 

non-verbal signals, providing valuable insight into team and social dynamics, 
ibe: 

no-prompt group, outperformed the prompt group, 799 

“normal” aging, reducing speed and efficiency of cognitive, perceptual, and 


psychomotor functions, 838 


normal curve, descriptive of the distribution of human traits, 311 


normative project, 478 

Norton Anthology of Poetry, biographies of poets listed in, 425 
notational algorithm, defined a search space of chemical structures, 88 
notational methods, externalizing plans during pre-writing, 418 
“notebooks of the mind,” 414 


novel activities or interactions, requiring effortful conscious thought, 116 


novel combination, of diverse bits of information, 813 


novel design decisions, correlation with triple-mode periods and the 
occurrence of, 379 
novel information, strong demands in handling, 223 


novel problems 


applying knowledge and procedures to, 195 
randomly generating moves and testing for effectiveness, 796 
novel situations, information encountered in, 774 


novelists, work sessions, 422 


novelty, degree of as person dependent, 116 


novice(s) 
applying explicit rules, 23 


attention directed toward technical, low-level aspects, 540 


avoiding gross mistakes, 752 
ball flight and, 685 


concentrated on the visual composition, 385 


dealing with the chess board in a piece-by-piece manner, 69 
developing SA, 720 
factors affecting SA, 719-720 


finding bottom-up strategies useful, 587 


having more scattered information search patterns, 717 
inability to access knowledge, 71 

initiated into a domain of skilled practice, 21 
interpreting scenes of fires, 71 


knowing the basics of a domain, 315 


lack of domain-specific knowledge in, 248 


not having a vast knowledge base, 234 
not realizing what information to seek out, 720 


organizing the perceived world according to “surface similarities,” 364 


outperformed experts with violated location information, 364 


performance of enhanced, 794 
performing well in sports, 802 
problems building SA, 735 


profiting from segmentations of animations, 799 


reasoning by analogy in chess, 606 
recalled about five chess pieces, 68 
showing similarities to expert ways of being, 36 


thinking about presented problems, 66 


understanding displayed by performing complex tasks, 33 
novice cognition, 64 
novice groupings, organized by salient objects, 69 
novice learners 
benefiting from explicit support during problem-solving, 800 
benefiting from step-by-step procedures, 806 


dealing with unfamiliar problems, 804 


explicit external instruction, 805 


instructional methods for, 797 
using search-based problem-solving approaches, 805 
novice pilots. See also pilots 


constrained by working memory, 717, 725, 730 


with low or moderate SA, 728 
overloaded by tasks, 729 
novice platoon leaders, 732, 733 


novice recall, not capturing basic game-advancing, 69 
Novum Organum (Bacon), 6 

nuclear power operators, 186 

nucleotide building blocks, of DNA, 823 


nucleus caudate, 243 


“nuisance variable,” in experiments, 67 
null moves, in chess, 605 


numbers, memorizing, 622, 700—701 


numeracy. See also statistical numeracy 
exerting a direct effect on affective responses, 496 
focus on the acquired skill, 477 


measured with the Berlin Numeracy Components Tests, 489 


out-predicting fluid intelligence, 493-494 


questions, 477 

related to fluid intelligence, 490 

scores predicting accuracy of disease risk interpretations, 477 
skills, 616 


standard for assessment, 477 


supporting skilled decision making, 496 


numerical abilities, explaining life outcomes, 485 


numerical capacity, in infants, 617 


numerical stimuli, experts “seeing” differently, 619 
numerosity, 617, 619 
nurses 

handovers, 119 


neonatal intensive care, 186 


patient length of stay and, 519 


nursing, reviews of, 206 


Oates, Joyce Carol, 425 
obituaries, employed in case studies, 293 
object(s) 
apparent shapes of perceived to be closer to their assumed “real” shape, 
582 
domain-specific knowledge of, 584 


drawing, 577 


identifying, 585 


recognizing, 241 


types of, 584 
object structure, 584, 590 


object-based attention, 155 


objective achievement, study of expertise focusing on, 13 


objective binary coding, of pixels, 588 


objective constraints, limiting expert performance, 143 


objective criteria, 295 


sports with and without, 273 


objective definitions, of expertise, 49-51 
objective exceptional performance, expertise demonstrated by, 49 
objective measures 

obtained, 301 


of a teacher’s relative performance, 747 


objective metrics, of drawing accuracy, 590 


objective performance 


measuring by improvements and outcomes, 747 


measuring in an age-independent manner, 750 


on representative tasks, 755 


objective representative performance, 760 


objective scoring systems, determining expertise, 314 
Objective Standardized Assessment of Technical Skills (OSATS), 340 
Objective Structured Clinical Examination (OSCE), 340 


objectivity, of scientists, 137 
observable behavior, 63, 304 


observable differences, between experts and novices, 15 


observation, novices coming to understand and form goal states, 114 


observational drawing 


entailing intense and prolonged perceptual engagement, 577 
not guided exclusively by visual information, 584 
psychological explanations for skill in, 580-586 


supporting more accurate, 584 


observational drawing skill, aspects of perception associated with, 580 


observational rendering, of realistic two-dimensional images, 577 
observational reports, containing errors and partial truths, 90 


observational studies, 271 


observational verification, of a forecaster’s model, 186 


observations 


available to appraise leader performance, 298 


of cases as structured or unstructured, 292 


of designers, 373 


as a method for measuring expert teams, 518 


studying self-regulation of learning, 277 
observers, 684 
obstacles, identified, 444 


obstetrics, gave up forceps in favor of the Caesarean section, 444 


“Occam’s razor,” 135 


occipital lobe, 156 


occipital visual areas, 240 

occipito-temporal junction (OTJ), 241 

occlusion techniques, 679 

occupation(s) 
differentiating in terms of service orientation and “moral community,” 128 
with goals, practices, and outcomes manifested situationally, 107 
interest and willingness to learn, 120 


as learned, not taught, 113 


occupational and practitioner interests, 131 


work of linked or similar becoming indistinct, 132 


occupational activity, manifested in concrete situations, 110 


occupational capacities 
developing, 120, 121 


historical perspective of the development of, 113 
learnt through participation, 113 


maintaining across working lives, 114 


occupational change (rationalization), 131 


occupational closure, resulting in monopoly, 130 
occupational competence 
described, 109 


explanatory account of, 122 


key premises for developing, 120 


occupational elements, learned through experience, 111 


occupational experiences, promoting depth of understanding, 108 


occupational expert, no such entity as, 111 


occupational expertise 
bases of, 109-113 


central to achieving societal and economic needs, 107 


cognitive and socio-cultural accounts of, 121 

described, 105, 109 

developing and sustaining through work activities and interactions, 
107-109, 113-116 

developing through everyday work activities and interactions, 105-122 

difficult to codify, 112 


domains of, 111 


predating provisions of education and training programs, 113 


premised in the circumstances of where practiced, 111 


represented as a product of personal history of experiences, 112 


skepticism about institutionalized, “schooled” activities, 106 


occupational goal states, gaining access to, 119 


occupational group, seeking a monopoly, 129 


occupational identity, group constructing, 131 


occupational knowledge 
domains of, 111-113 


making accessible, 119 


securing, 115 


“occupational level,” 225 


fs 


occupational performance, 111, 77 


occupational practice 
foundational domains of, 112 


identifying qualities of, 122 


learning, 118 


occupational roles, legal, 137 


occupational tasks, 109, 111 


occupational values or dispositions, derived socially, 110 


occupational workers, all having expertise, 129 


OCD, leading to dystonia-triggering behaviors, 564 


Ocean, Humphrey, 592 
ocular fixations of experts, 679 


offshore drillers, situation awareness in, 458 


offshore well control, simulations for research and training in, 465 


Ogasawara, Naofumi, 626 
“old” or “older,” no general consensus on, 835 
older adults 


benefiting less from training, 846 


expert performance in, 839-840 
increasing attentional costs exerted by simple bodily functions, 847 
maintaining high levels of skill, 849 


recruitment of symmetrically contralateral brain regions, 844 


rediscovery of as valuable participants in the work force, 849 
older amateur musicians’ rhythmic timing, 845 
older chess experts, 844 
older chess players. See also chess players 

diminishing return for cumulative purposeful practice, 846 


needing greater current practice than younger players, 843 


showing a second inflection point, 847 


older experts 
accomplishments presenting a puzzle, 835 


actively maintaining specific skills, 849 


compensating for age-related declines, 843 


increasing difficulty expected to learn new techniques, 846 


investing deliberate effort into the development of skills, 840 
maintaining performance into older age, 75, 835 


performance of exempted from age-related slowing, 838 


showing “normal” age-graded declines, 840 


stability of performance in, 840 
timing skills protected from age-related decline, 841 


undergoing decline from an initially higher baseline, 839 


older individuals, adopting compensatory mechanisms, 844 


older musicians. See also musicians 


advantages in auditory processing, 847 


compensating for decline in primary auditory cortices, 842 

older pianists. See also pianists 
counteracting losses in motor performance through practice, 540 
maintaining levels of performance for expert, 844 


maintenance practice by expert, 761 


performance correlated with the amount of practice alone for expert, 761 


years of “experience” compared to amount of deliberate practice for 


amateur, 846 


olfactory expertise, 237, 239 


olfactory experts, as rare, 238 


Olympic medal winners, on performance characteristics needed to become an 
expert, 275 


one-on-one instruction, 756 


on-field anticipatory performance, 689 


ongoing and desired performance, connection between, 542 
online methods of data collection and verification, 266 


online tutoring program, on using graphs, 496 


on-the-job expertise acquisition, 398 


ontogenetic development, 111 


ontogenetic ritualization, 121 


open information sharing, 403 


open sports, 705 

openings, in chess, 602 
openness to experience, 391 
operas, 314, 315 


“operating conditions,” of science, 136 


operational domain, stress, fatigue, and workload occurring as a function of, 
719 


operational experts, working independently, 185 


operations component, of numeracy, 488 


operators. See nuclear power operators; vehicle operators 
“opportunism,” 384 


“opportunistic” behavior, of designers, 383 


optimal designs, including multiple assessments/timepoints, 263 


optimal environments, for reaching high levels of expertise, 625 
optimization methods, 479 


options analysis, 179 


options and outcomes, feeling the weight of various, 483 


oral history, employed in case studies, 293 
oral mode, lexical coverage in, 636 


orbitofrontal cortex, olfactory expertise and, 239 


Orca mothers, 53 


orchestra conductors, 538, 557 


orchestra members, performance of, 747 


order, imposing on uncertainty, 375 


order of operations, mature deployment of, 155 


ordinary occupation, expertise as the base of, 142 


organic macromolecules, 823 


organisms, responding to the present by using the past, 820 


organizational climate, 522, 523 


organizational professionalism, 131 


organizational psychology, 838 


organizational routines, 444 


organizations 


facing highly competitive markets and continued change, 510 


having to meet to be professionally successful, 106 


implementing cooperative reward structures, 511 


virtuality in as growing, 516 


organized/formal “practice,” contrasted to “play,” 265 


orientation, found in taxi drivers, 245 


original approach, 813 


Orosco, José, 827 


outcome variables, 357 
outcomes 


assessment of, 295 


cases appraised with respect to, 297 


not identical for all individuals, 551 
of teaching affected by numerous factors, 445 


outline, preparing, 418 


output, of highly influential works, 314 


outstanding achievement, relation between age and, 320 


outstanding designers, 373 


outstanding performance, resting on specific mechanisms, 839 


overall memory, compared to selective memory, 708 


overlearned sensory motor programs, 564 
over-pattern matching, as a risk, 724 
“overtraining,” 318 


oxygen, maintaining normal race pace, 749 


oxytocin, increased release of, 553 


Oz display, for pilots, 367 


pain syndromes, neural studies of, 566 


painting, using modern materials for, 827 


pairwise comparison, as a measurement of SMM, 512 


paradigm, 166, 642. See also problem-solving model 


paradigmatic decision domains, 486 


paradigmatic decision making competency, 488 


parahippocampal area, 238 
parahippocampal gyrus (PHG), 240, 242, 243, 244 


parahippocampal place area, 157 


parents 


designing optimal environments, 625 
helping children plan practice and to identify errors and sustain attention, 
756 
rejecting consensual medical evidence, 22 
parietal cortex, 247 
parietal lobe, 555 


Parsons, on professions, 128 


partial structure, of a solution space, 378 


participant and non-participant observation methods, 373 


participant follow-up, 281 


participant observation research, 260 


participant-observer studies, 374 


participants 
attaining eminence in some domain, 313 
attrition of, 281 


in cultural activities, 435 


giving similar tasks many times, 700 


inferring reasons for decisions, 195 


seeking to attain an acceptable level of performance in a new activity, 393 


participation, predicted theoretical influence, 301 


participation rates, in chess, 608 
Pascal, Blaise, 479 


passive learning, idea of, 393 


past experiences, 265 


patent trends, throughout Edison’s career, 296 


path independence, 487 


pathophysiological explanations, demonstrating the intermediate effect, 803 


pathophysiological mechanisms, descriptions of, 337 


paths, to expert performance, 76 
pathways, as means of organizing learning, 118 
patients 

care of, 119 

length of stay, 519 


with musician’s dystonia, 568 
outcomes, 342, 747 


presentations, 344 


satisfaction ratings, 747 
pattern(s). See also chunks 
allowing experts to retrieve suitable actions from memory, 11 


with associated action patterns, 115 


of chess pieces, 697 
as configurations of cues, 455 


defined across multiple data types, 455 


experts extracting, 451 


identifying, 661 
mediating improvement in skill, 11 


much of language composed of, 643 


occurring within the sport domain, 684 


order of presenting, 683 
slow acquisition of more complex and refined in LTM, 697 


pattern matching, 722 


experts’ ability to use, 456 
facility with significantly correlated with SA, 722 
to learned schema, 722 


skills coming into play, 725 


some people better at, 722 


pattern recall, 682, 683 


pattern recognition 


closely related to pattern recall, 682 


delineating in entrepreneurs, 392 


essential information needed to guide, 684 
probing a performer’s ability to recognize rapidly, 682 


providing a better representational structure, 599 


role in move selection in chess, 598 


underpinning skill in chess, 603 
Pauling, Linus, 823 
PDSA (Plan/Do/Study/Act/) cycles, 444 
peak, specific location of, 320 
peak height velocity (PHV), 276 
peak performance, 273, 750 


peak rating, predicting in chess, 608 

peak skill development, pattern of, 53 

pear, drawing from observation, 577 

pedagogic practices, as part of everyday work activities, 117 


pedagogical content knowledge, 441 


pedaling, as a highly refined skill, 561 
peer reviews, 137-138 
peer-nominations, by professionals in the same domain, 4 


peer-report method, for measuring expert teams, 518 


peers 
assessing superior outcomes of experts, 746 
described, 137 


relying on secondary and non-specific information, 746 


people, as goal driven, 718 


people moving, across our planet, 356—357 

perceived system information, 721 

perceiving body, 34 

perceiving visual stimuli, relationship with accurate drawing, 583 


perception. See also hazard perception; visual perception 
centrality of, 152 


changing across development, 151 


collaborating with action, 35 
complex, 152 
concentrating attention during, 816 


conscious, 581 


directing attention during, 3816 


everyday, 577 
in expertise, 151—160 


expert-level, 157 


hypothetical relations with drawing, 579 


imagined objects interfering with, 157 
influenced by experiences, 151 


intertwined with development of expertise, 151 


involving “bottom-up” and “top-down” processing, 580 


of mathematical experts, 619-620 
of music, 550, 554 


never separate from action, 151 


of relevant information from the environment, 715 

risk, 486, 487 

role in the acquisition and deployment of expertise, 152 
sensory, 566, 567 

shogi, 243 


skilled, 274 
skilled object, 242 
tuning, 152 


NJ 


perception and action, connection between, 151, 15 


Perception research category, for transportation, 357 
perceptual advantages, of artists as real, 580 


perceptual and memory mechanisms, underpinning skilled performance, 597 


perceptual and motor skills, art and art education and, 592 


perceptual constancies, 581 


perceptual diagnosis, limitation to a domain, 67 


perceptual discovery learning, 689 


perceptual discrimination and pattern perception, 454 


perceptual encoding, 581 
perceptual experience 


in the course of the acquisition of expertise, 151 


expertise changing low-level, 157—160 


perceptual expertise, 235-239 
categories of, 151 


rarely necessary to complete skilled conceptual tasks, 154 


structural brain changes in, 239 


perceptual information, differential use of, 679—680 


perceptual judgment errors, positive correlations with drawing errors, 581 


perceptual learning. See also learning 


acquiring, 454 

activities as “the very foundation of intelligence,” 774 
aspects relevant to visual art, 578 

impact on higher level mathematical skills, 619 
theorists, 680 


perceptual mechanisms 
deployment of, 152 
impact of, 151 


sequential operations of, 153 


perceptual memories, differences in, 362 
perceptual processes 


across different domains, 578 


factors influencing, 716 


pure, 152 


perceptual processing, 235 


perceptual regions, 157 


perceptual routines, role in expertise, 152—156 


perceptual sequences, early recognition of, 681 


perceptual skill differences, tracing by examining eye movements, 598 


perceptual skills 


learning appropriate, 152 


providing a critical edge, 455 
perceptual speed and psychomotor abilities, predictive validity for task 


performance, 222 


perceptual strategies, 153 


perceptual task, facilitating creative thinking, 816 
perceptual training. See also training 


accelerating the rate at which anticipatory skills are required, 688 


producing dramatic gains increasing accuracy and reducing response times, 
620 
requiring learners to make anticipatory judgments while watching 


occluded video clips, 688 


perceptual-cognitive skills 


acquired in one domain being exploiting in other domains, 660 
exploring interactions between, 663—665 


importance of differing across task constrains, 664 


relationships with practice activities, 658—660 


transferring across related sports, 660 


underpinning anticipation and decision making, 653, 669 


perceptual-motor activity, 152 


perceptual-motor adaptation, of musicians, 543 


perceptual-motor integration themes, 591 
perceptual-motor procedures, 77 


perceptual-motor processes, 155, 759 


perceptual-motor processing networks, conceptual tasks offloaded onto, 152 


perceptual-motor routines, aligning with existing representations, 156 


perceptual-motor skills, expertise in, 67 


percussionists, 543 
perfectionism 
attitude of, 420 


characterizing writer’s block, 422 


performance. See also deliberate performance; expert performance; superior 


performance 
accuracy of, 62 


assessing, 44, 747 


attaining a functional level of, 75 
attributes, characteristics, competencies and skills underpinning, 666 
automatized and transformed through practice to skilled, 115 


averaging across multiple experts, 316 


changing as a function of age, 314 


in clinical practice, 344 
constraints of in music, 544 
control and reproducibility of, 759 
development of, 275, 750—752 


differences between usual and maximal, 847 


as directly observed behavior, 667 
distinguished from learning, 667 


effectiveness of practice activities to improve, 759-763 


effects of accumulated experience on attained level of, 752-754 


effortful exertion to improve, 424 

as expertise, 2 1—22 

of experts, 67 

fluctuations idiosyncratic to that particular person, 316 
fluctuations of, 228 


generating virtually automatically with a minimal amount of effort, 753 


high level inherent in the notion of being expert, 36 
hypothetical development of, 272 
improving, 544, 668 


indicators, 762 


interdependence of, 220—221 


IQ, age-graded declines in, 837 


limited by more basic and narrow abilities, 222 


maintained in old age for speeded expertise tasks, 843 


measures in historiometric inquiries, 314 
measures taken in the simulation laboratory and in patient care contexts, 
342 


measures taken under artificial laboratory conditions, 221 


measuring, 272, 700 


of a musical piece, 539 


not solely dependent on the efforts of the individual performer, 220 


outcome, measuring between various sports, 285 


placing within context, 274 


reaching a plateau, 752, 754 
related to the difficulty level of the sight-reading task, 707 


representative in domains, 12 
reproducible, 4, 23, 750—752 


restriction of range of, 220 


studying the development of, 274—276 
success depending on the actions or behaviors of others, or on 


environmental influences, 220 


systems used to collect data, 258 
tests yielding predictive scores differing at different ages, 752 
tracing the development of, 746—748 


performance characteristics 
specific, 274-276 
studying general, 276—277 


subject to change to a much larger degree, 278 


unraveling, 272 


performance domains, 50 
performance means, over task practice, 219 


performance metrics, related to quality of precision movements, 340 


performance modeling, 777 
performatory activities, producing an expected result, 775 
performers 
comparison of young to already elite counterparts, 265 
diaries of, 259 


modifying behavior when observed, 260 


reaching highest level of performance in mid-to-late 20s, 750 
Periodic Table of Expertises, 24, 25—26 


periodization, 264 


peripheral vision, 686 
person relative motion information, facilitating pattern recognition, 661 


personal domains, of occupational knowledge, 109, 111-113, 115 


personal epistemologies, 117, 120-121, 122 


personal experience, affecting perceptual experience, 151 


personal mediation, of individuals, 109 


“personal professional theories,” 106 


personality 
of chess players, 607 
diversity influencing team relationships, 515 


individual-difference variables like, 592 


personality traits. See also affective traits 


measures correlating with success, 224 


as predictors of change over time in leadership efficacy, 520 


personality variables, 390, 391 


personalized CMC, positive effect on extra-group network size and structural 
holes, 517 
personalized training tasks, 806 


personally oriented heuristic deliberation, 483 


person-dependent process, of learning and developing domains of 
occupational knowledge, 121 

person—environment coupling, as the unit of analysis, 771 

persons. See also individuals 


having the opportunity to make their own decisions, 498 


persons and world, inescapable relation between, 34, 38 
“persons-in-the-shadow,” role of, 107 


persuasion, inherently intersubjective, 403 


“the phantom plateau,” examples of, 194 


phatic communion, 645 
PHaVE List for phrasal verbs, 647 
PhD candidates, productive writing by, 420 


phenomenological account, of improvising jazz on the piano, 540 


phenomenological approach, investigating transition experiences of 
developing athletes, 262 

phenomenology, founder of, 34 

PHG. See parahippocampal gyrus (PHG 

Philosophical Transactions of the Royal Society in 1665, 137 


phrasal verbs, 644 


Phrasal Verbs Machine application, 648 


PHRASE List for non-transparent formulaic sequences, 647 


physical actions, performing more automatically, 722 
physical and mental resources, showing differential age-related changes, 845 


physical capacities, tending to peak around the age of thirty, 273 


physical demands, of musical instruments, 564 
physical environments, 154, 466 

physical exercise program, during space flights, 754 
physical growth spurt, 276 


physical limitations, on performance at high levels of expertise, 217 


physical or cognitive abilities, small differences exaggerated as performers 
reach elite levels, 43 


physical process, changing, 178 


physical setting, for design thoughts, 380 


physical skills 
developing required, 729 


non-conscious levels becoming automated, 114 


physical tasks, automaticity of, 722 


physical traits, 213 
physical traumata, 566 


physicians. See also medical doctors 


accuracy in diagnosis after hearing the chief complaint, 332 


clinical judgment not cultivated using a deliberate-practice approach, 779 

exceptional identifying common themes, 345 

experienced, outperformed medical students on diagnostic performance, 
802 

expert having a more differentiated knowledge of diseases, 66 


expert representing diseases as an extended process, 71 


full scope of expertise, 780 


looking for negative reactions to medications, 724 


peer nomination of, 344 
referring to comfort level while deciding to continue with laparoscopic 


surgery, 464 


physics, studied by thinking aloud procedure, 63 
physics problem solvers, expert and novice sorted groups of problems, 69 
physics problems, from introductory college courses, 195 


physiological adaptations, of musicians, 542—543 


physiological characteristics, of adults, 754 


physiological factors, limiting performance of selected individuals, 543 
physiological fitness, of astronauts reduced during low-gravity space travel, 
754 


physiological limits, of musical performance, 544 


physiology and anatomy, adapting to particular types of training and 
experience, 752 
Piaf, Edith, 552 


pianists. See also expert pianists; older pianists 
actual practice behavior of professional, 207 


central sulcus depth in, 560 
cramp, 563 
deliberate practice for, 757 


drawing on a whole set of complex skills, 550 
early start of training, 536 
effect of expertise on life-long plasticity for middle-aged, 560 


experiences from a very young age, 554 


expert and amateur of different ages, 843 
hand-independence in rhythmic timing, 837 


movement inhibition following a stop signal in, 567 


observing video sequences of a moving hand at the piano, 558 


obtaining an objective measure of abilities, 562 
older professional showing normal age-related declines, 843 


optimized functionality of neural structures, 563 


practice hours accumulated by amateur, 654 


predicting regularity in scale playing of adult, 540 
requiring increased sensitive tactile discrimination, 543 


right hand more frequently involved, 564 


showed more gray matter in specific regions, 562 
showed no extra activation to accommodate task complexity, 841 
summary of the results of the study on, 562 


tapping faster and more accurately than controls, 543 


targeting the middle putamen in the basal ganglia, 561 


tending to be practice fanatics, 537 


trained to perform memorized pieces, 746 


piano, 40 million Chinese seriously playing now, 544 


piano lessons, older participants showing improvements, 848 
piano playing 


causal relationship with enlargement of the corpus callosum, 560 


practice time as the only predictor of variation in “temporal evenness” in 
playing scales, 262 


role of pedaling in, 561 
skill level of, 561 


starting students with basic techniques, 751 


piano tones, responses to in musicians, 557 
Picasso, Pablo, 625, 826, 831 


pieces (musical) 


deemed unplayable at the time of their composition, 544 
learning new and working on difficult spots as most effortful and not 
enjoyable, 538 


polishing of, 539 
PIFS (Practical Intelligence for School) program, 783—784 


pilot error, 727 


Pilot-in-the-plane principle, 395 
pilots. See also civilian aircraft pilots; experienced pilots; expert pilots; 


military pilots; novice pilots 
anticipating engine failures, 723 


better, gathered more information, 725 


better, more attuned to constraints, 725 
with better SA, 725 


developing mental models, 720 


experienced, better at directing attention to critical cues, 721 
experienced, having more automated skill and a more refined mental 
model, 717 


experienced, outperformed the inexperienced, 728 


glider, performed better on a divided attention task, 725 
keeping track of many factors, 715 


learning to communicate with ATC and to scan their instruments, 723 


less experienced, accounting for the majority of aviation accidents and 
fatalities, 726 

less experienced, flew more stable approaches in black hole conditions, 
360 

less experienced, having problems dealing with distractions and high 
workload, 728 


line, dealt more at the level of comprehension (Level 2 SA), 726 


more experienced, actively modified the situation, 360 


older, gaining less from simulator training, 846 


poorer, interpreting cues inappropriately and under estimating risk, 725 
projection taking up a considerable portion of time, 734 


risk assessment during a hypothetical windshear situation at takeoff, 464 


running procedural checklists to make sure they check each item, 723 
SA and expertise in, 726—730 
SA errors, 715 


selected through a number of screening tests and highly trained, 726 


simulator training affecting gains for a variety of tasks, 846 
societal expectations for, 109 
trainee, permitted to solo, but not allowed to take passengers, 357 


using simulation data to examine decision processes in, 465 


piriform cortex, enlarged in perfumers, 239 


piriform gyrus, olfactory expertise and, 239 
“the pitch” 


consisting of an entrepreneur targeting particular stakeholders, 399 


nomenclature of, 399 


pitches (musical) 
accurate discrimination for, 543 


naming without a reference pitch, 551 


perception absent or highly deficient, 552 


pitfalls, as artifacts of technology, 466 
PL (perceptual learning), 620 
place of performance, musicians adjusting performance to, 73 
Plan/Do/Study/Act cycles, of improvement science, 446 
planning, 179, 277 

addressing, 61 

for the future, 433 

goals of, 462 

increased with higher skill, 206 


reducing mistakes and failures in chess, 702 


scheduling systems and, 97 


skills in the sciences, 294 


for writing, 414 
plasticity. See also brain plasticity 
adaptability and, 660 


coping with variation and change, 41 
involving the sensitivity of systems, 42 
plateaus 


as a consequence of strategies used, 194 


in improvements, 5393 


performance reaching, 752, 754—757 


in professional domains, 752 


platinum wire, configured as a possible burner, 818 


platoon leaders 
as the entry-level officer position, 730 
gathering information from the battlefield environment, 731 


inexperienced, having difficulties with forming good SA, 733 


new, having difficulty projecting a likely enemy course of action, 733 


play and exploration, primary competencies and, 43 
players 
positioning, 664 


representing mentally where teammates and opposing players are, 706 


playing technique, alteration of at a later age, 566 
PMd premotor areas, engaged in abacus experts, 241 
pMTG (posterior MTG) 

activated by chess objects, 243 


engaged in both experts and novices, 243 
sending processed information about body motion, 247 


smaller in chess experts than in non-players, 245 


poetry, scaffolding for forms of, 42 
poetry composition, {MRI study of, 418 
poets, 418, 422, 426 

Poincaré, Henri, 813, 814 

point in life, 551 


point-light displays, reducing complex visual displays, 680 
points-of-view, observing teams from selected, 168 

poker, participants confronting challenging situations, 203 
Polanyi, Michael, 92, 772 


Polgar sisters, 609, 625 
policy-capturing models, success of applying, 777 


political fragmentation, 318 


political leaders, assessed with respect to assassination attempts and 
successful assassinations, 295 

political tactics, variables bearing on, 299 

Pollock, Jackson, 826 

polychronicity, 513 


polymodal association cortices, 558 


polyrhythms, piano players mastering, 751 


polysemous words, 642 


pop music, with improvised structures, 566 


“popular obedience to the law,” as the sole source of legitimate rule, 131 


popular understanding, 25 


population trends, multilevel model describing, 280 


porcelain, manufacturing, 117 


positive domain-general cognitive outcomes, 847 


positive knowledge, assisting to effectively achieve goals, 115 


positive monotonic function, resembling a learning curve, 311 


positron emission tomography (PET), 415 


post hoc ergo propter hoc errors, 266 
postcentral gyrus, 562 


posterior cingulate, 243 


posterior cingulate gyrus, 237 
posterior cuneus at the medial side of the brain, 243 
posterior hippocampus, 240 


enlarged for taxi drivers, 245 


posterior parietal area, 555 


posterior parts of the cingulate, 244 
posterior precuneus, 243 


posterior superior temporal gyrus, 559 


post-peak decline, varying according to the particular domain, 320 


post-training brain scan, for taxi drivers, 245 
postural cues, 661, 663 


postural information, access to, 664 


potential interaction, between different perceptual-cognitive skills, 663 


potters, placing hands upon those of novice potters, 117 
potter’s wheel, 118 
poured paintings, of Jackson Pollock, 826-827 


power 
of elites, 139 
in knowledge, 87 

power and control, linking expertise to, 21 


power athletes, 248 


powerboat drivers, 359 

powered flight, Wright brothers’ development of, 300 

PPIK (intelligence-as-Process, Personality, Interests, and Intelligence-as- 
Knowledge), 227, 228 


practical decision-relevance, inductive logic, 479 


practical inductive reasoning, decision makers skilled in, 483 


practical intelligence 


as acquisition and use of tacit knowledge, 774—77 
commonality among measures of general tacit knowledge, 775 
defined, 774 

developing, 780—784 


differences in appearing during the middle school years, 783 


increasing prediction of both academic and extracurricular success, 779 
leveraging conceptual alignment with ecological theories, 784 
practical intelligence and tacit knowledge, theory of, 771, 784, 785 
Practical Intelligence for School (PIFS) program, 783—784 


practical know-how, essential to expertise, 781 
practical problems, applying tacit knowledge to, 774 
practical skills, research on high levels of, 65 
practically intelligent behavior, addressing directly, 780 


practice. See also deliberate practice; effective practice; purposeful practice; 


team practice 
age-related constraints on improvement through, 846-847 


all types not equally effective in improving performance, 760 


amount not correlated with performance for beginning music students, 760 


amount of, significantly related to level of performance, 537 


based on mentoring or expert instructional guidance, 464 


characteristics of when the performer is in control, 260 


comparing methods, 759 
data, 265 
differences between different types of, 709 


durations, 537 


in effectuation, 398 


enhancing the quality of, 538 


evaluations of the micro-structure of, 258 


experiences, 654—656 
for expert calculators, 625 
with feedback improving memory performance, 698 


as a function of time in season, 264 


gauging amounts of, 264 


high levels of seen in creative writers, 424 


increasing performance through, 536-539 
investing the time, 536—538 


lacking inherent enjoyment, 538 


as Maintenance work, 539 
methods for assessing, 759 


motivated by a greater objective to improve performance, 398 


permitting learners to acquire key, functional aspects of a task in a more 
transferable manner, 343 
play related activities in sport and, 659 


quality of, 258 


rate of intensity declining after age 50, 842 


reductions in leading to inferior performance, 763 


related to improvements in working memory support of chess playing, 703 


relating to anticipatory skill, 687 


relation to attained performance, 761 


remaining the key to practical accomplishment, 28 
required to become socially competent, 393 


required to develop skills and execute these complex tasks, 551 


role and manifestation not identical in all musical genres, 536 
schedules, 260, 668 
settings, 121 


systematic observation of, 260 


taking into account current ways of being, 36 
with zeal, 654 

practice activities 
current amount of, 393 


differential effectiveness of, 759-763 


hours accumulated by violinists, 654 
identifying types of leading to specific skills, 656 
improving specific aspects in a protected environment, 75 


individually determined as deliberate practice, 265 


meeting several of the criteria for deliberate practice, 758 
modifying mechanisms mediating performance, 709 


negatively related to mortality in the coronary care unit, 345 


of older expert pianists, 761 


outside the target domain, 659 


versus “play” activities, 265 


relationships with specific perceptual-cognitive skills, 658—660 


relevant differing across domains, 393 


taking account of the performer’s current skill level, 398 
testing on a micro-level, 656 


types of, 759 


underpinning anticipation and decision making, 665 


practice activity data, collecting and assessing, 257—266 
practice alone, changes in the effects of, 760—761 
practice and play related activities in sport, leading to specific adaptations in 


memory, 660 


practice conditions, 667, 668 


practice curriculum, 1 17—119 
practice environments 


developing, 18 


randomness and unpredictability of, 260 


“practice guidelines,” prescribing the “correct” approach, 347 
practice histories 


analyses of successful and less successful individuals, 262 


gaining across multiple age groups, 266 


information, 264 


of sub-experts in music, 537 


various approaches to, having strengths and weaknesses, 266 


practice history profiles (the macro level), 654—656, 658 
practice hours 


in an Academy setting as quite fixed, 659 


during childhood lower than in adolescence and adulthood, 560 


cumulative, 561 
practice pedagogies, 119-120, 121 


practice periodization, in sport, 656 
practice tasks 
in entrepreneurship, 398 
in industrial process control compared with law, 402 


involving teams and constellations of individuals, 401 


selected by a teacher, 424 


practice time, using effectively, 760 
practice-based activities and interactions, ignoring, 113 


practice-based adaptations, 665 


practice-based experiences, 112 


practice-based expertise, 53 


practice-based instructional support, 106 


practice-based learning, 108 


practice-plasticity-processes model, 610 
practices 


of occupational knowledge, 110 


of teachers across the highest achieving countries, 439 


practice/training blogs, 260 


practicing, as an effortful activity and a skill, 539 
pragmatic leadership style, 299 

Prairie House, design Wright developed, 826 
precedents, 381—382 

precentral gyrus, 555 


precise information transfer, 645 


precision, accounting of, 214 


precocious impact, 320 


precuneus, 157, 245 


precursor, 363, 364 


prediction 
of expert performance, 213, 216-221 


as a key component of how experts overcome neural delays, 685 
key to criterion-related validity, 215 


prediction period, shorter as more accurate, 666 


predictive and anticipatory behavior, role in expert performance, 677 


predictive gaze behavior, creating time by, 684—686 
predictive performance 


of domain-specific experts and novices, 680 


questioning the best way to improve, 688 


predictive power, of conventional numeracy, 489 


predictive reasoning, 394 


predictive saccade (a fast eye movement), produced by cricket batsmen, 685 


predictive strategies, useful in dealing with risk, 394 
predictive validity, 216 

accumulated hours of practice, 537 

estimating for the SAT, 216 


of a measurement tool, 264 


predictor measures, 220 


predictors 


criteria and, 223 


of expertise in medicine, 345 


of expertise in surgery, 345—346 
predisposing factors, 564, 565 


pre-existing individual differences, from a large Swedish twin study, 551 


preferred leadership style, measure of, 300 


preflight preparation, 726 
prefrontal cortex, exchanging information with, 247 


prefrontal regions of the brain, 838 


prehistoric art, archaeological evidence of, 576 


premature automaticity, 447 
premature closure, 334, 719 


pre-mission planning, 724, 726 


premorbid OCD, found in dystonia patients, 565 


premotor and motor areas, engaging, 243 
premotor area (PMA), 247, 555, 556 
premotor cortex (PMd), 241 


preparation, long periods of, 820 


preparatory (undergraduate) program, 6 


preparatory nature, activities not as enjoyable, 538 


pre-practice observation, 118 


prescriptions, versus descriptions, 265—266 
prescriptive feedback, providing sparingly, 668 
prescriptive project, 478 


present tense, in a journal article, 419 


preserved differentiation accounts, 839 


pressure, deteriorations in performance occurring under, 803 

prestige, accruing to those who develop expertise, 45 

prestige bias, in other human learners, 45 

pre-supplementary motor (pre-SMA) and rostral pre-motor regions, 
activating, 841 

prewriting, 415, 418, 421 


priests, in pre-modern societies, 127 


primary (direct) market research, results for entrepreneurship exercise, 406 


primary and secondary auditory areas (Al, A2), 554 
primary auditory area, 554 


primary auditory cortex (Heschl’s gyrus), in professional musicians, 559 


“primary generator,” instantiating a solution concept, 375 


primary knowledge, 795 


primary mental abilities, 836 


primary motor area (M1), 555, 556 


primary motor cortex, 560 


primary olfactory areas (amygdala), 239 
primary or innate abilities, determining expertise development, 837 


primary school students, benefiting from feedback, 800 


primary sensory-motor and inferior parietal cortices, connectivity changes in, 
568 
primary somatosensory area (S1), 555 


primary source knowledge, 25 


primes, man with an IQ of 67 generating and factorizing, 619 
principal component analysis, 279 
principal solution concept, 382 


principles, of occupational knowledge, 110 


principles and practices, underpinning adaption, 112 


prior ability profiles, 848 


prior examples, abstracting general principles from, 381 


prior knowledge and expectations, role in SA, 718 


private interests, pursuit of, 130 


private pilot’s license, 357 
privately held knowledge, 93 
privileged networks of expertise, 22 


proactive leader sense-making, 456 


probabilistic functionalism, Egon Brunswik’s theory of, 777 
probabilities 

associated with events or outcomes of interest, 91 

high, 662 


probability component, of numeracy, 488 


probability expressions, misinterpretations of, 494 
probability statements, 91 
probability theory, 479, 485 


probe questions, 176 


problem(s) 


attending to important aspects of, 152 


common, 205 
definition, 386 


exploring complex, 783 


interactive discussions of specific, 92 


simulating the complexity of actual, 777 
tackling in a “difficult” way, 376 
of writers, 414 


problem and solution, as co-evolving, 378 


problem formulation aspects, of design behavior, 377 


“problem frame,” establishment of, 382 
problem framing 

activity of, 385 

by designers, 376-377 
“problem paradigm,” of the designer, 377 


“problem scoping,” successful design behavior based on, 386 


problem setting, characteristic of professional reflective practice, 376 


problem solvers 
characteristics of expert, 86 
expert and novice in physics, 69 


successful, 462 


problem solving 
all breeds of dogs showing about the same average level of performance, 
52 
behavior, 375 


classic work on general, 62 


compared to designing, 376 

computational models of, 605 

computer programs modeling, 60 

“exhibiting” some of the characteristics of, 62 

exploring the power of knowledge in, 87 

at high levels, 86 

higher level theoretical constructs for, 61 

information processing models of, 11 

kinds and organizations of knowledge critical in successful, 333 

making contact with internal mechanisms of great extent and complexity, 
61 

needed by experts, 774 


in political science by experts and novices, 66 

psychological theory of, 63 

requiring imaginative groping towards a solution guided by tacit clues, 774 
studied using blindfold chess, 605 


variables bearing on, 299 


in virtually all domains of expertise, 541 


problem solving condition, 798 


problem solving operations, 72 
problem solving processes, in chess, 602—605 
problem space 


of assessing clients and responding to requests, 112 


finding efficient means to search, 86 
of an occupation, 111 


in which teachers work, 432 


“problem structuring” activities, at the beginning of the design task, 377 


problem subcomponents, 155 


problematic situation, applying one’s expertise to through repeated search, 
832 


“problem-based learning,” creation of, 65 


problem-solution pair, 378, 381 

problem-solving architecture. See problem-solving model 
problem-solving knowledge, 91 

problem-solving methods, 93, 99 

problem-solving model, 99 

problem-specific transfer, 821 

procedural content knowledge (knowing), 342 
procedural knowledge, 106, 120, 773 


proceduralization, process of, 121 


procedure and regulation compliance, 98 
Proceedings of the Innovative Applications of Artificial Intelligence (IAAT) 
conference, 98 
process(es) 
everyday or practice-based, 106 


fostering an output (team performance), 508 


of writing, 415 


process control analyst, 402 


process models 
of problem-solving in chess, 605 


understanding expertise in chess, 598 


process monitoring and control systems, analyzing real-time data, 96 
process tracing, 207 


process vs. product oriented examples, 798—799 


processing 


mechanisms underlying expert performance, 836 


occurring rapidly, 718 
restrictions in working memory, 797 
speed of, 836, 838 


process-oriented 


vs. product-oriented worked examples, 801 
worked examples, 798 
procrastination, 420, 421, 422 


prodigies 
as born or made in chess, 606-608 
defined, 625 
described, 751 


distinguishing in mathematics, 616 


possibility of accelerating typical development as unknown, 540 
product, imaginary, 405—406 
product and services, personalized recommendations of, 98 


product plans, employed in case studies, 293 


production rules, 11, 87 
production systems, building psychological simulations, 88 


“productive confusion,” inducing, 435 


productive knowledge, 642 


productive struggle, producing learning, 439 
productive thinking and insight, 817 


productivity, in the final years of a career, 320 


product-oriented worked examples, 798 


profession(s) 
as an alternative approach to the hierarchy of bureaucratic organizations, 
129 


“capturing” states and negotiating “regulative bargains” with states, 129 


developing agreed terminology, 132 
difficulties defining, 128 


as the knowledge-based category of service occupations, 130 


needing to close markets, 130 


science as, 137 


as the structural, occupational and institutional arrangements for work, 130 


professional actors, deep encoding by, 63 


professional associations 


akin to communities of practice, 781 


certifying acceptable performance and the permission to practice, 9 


membership of, 130 


professional communication, 413 


professional competence, age-related reductions affecting, 838 
professional decision making, expertise in, 66 


professional design, expertise in, 372—386 


professional designers, think-aloud verbalizations by, 206 


professional development 
focusing on, 66 
in teaching, 443 


professional development program, 448 


professional domains, evidence for plateaus, 752 
professional engagement, 142 


professional experience, 346, 846 


professional expertise 
age-comparative studies of, 840 
consisting of two related dimensions, 142 
different types of, 15 
professional groups 
conceptualized in terms of exclusionary social closure in the marketplace, 
129 
sociology of, 128 


1 


professional interpreters, 704, 70 


professional mathematicians, 628 


professional musicians. See also musicians 


aged 50 to 77 years of age, 848 


larger middle section of the corpus callosum, 560 
most suffering from medical problems, 544 
outperformed controls, 848 


poor performance on tests of musical talent, 836 


professional project, 129 


professional scientific identities, defining in science, 137 


professional singers. See singers 
professional skill or expertise, in domains with extreme demands on speed 


and accuracy, 839 


professional values and identities, development in workers of shared, 130 
professional work. See also work 


differentiating from human work in general, 132 


epistemology of, 132-134 


speed-up of, 96 
professional writers. See also writers 
characteristics of, 416-422 


crafting knowledge to the needs of a specific audience, 423 


domain expertise of, 419 
drawing upon language patterns stored in long-term memory, 424 


engaging the reader in deep comprehension, 416 


ethnographic survey studies of, 424 


facing frustrations, anxieties, and other negative emotions, 421 
improving writing skills, 424 
many specific kinds of, 426 


needing to continue training through an apprenticeship, 425 


rapidly assembling a first draft, 419 
reviewing texts for the “form or shape of their argument,” 423 


as self-motivated, 420 


professional writing 
defining, 413-414 
deliberate practice in, 75 


reflecting expertise in a particular genre and domain, 426 


professional writing expertise, 413—426 


professional—client relationship, in science, 136 


professionalism 


appeal as a mechanism of occupational change, 132 


depending on socio-cognitive competence, 142 


dominating in fields where new standards for best practice need to be 
established, 143 
force for stability and freedom, 128 


implying the importance of trust in economic relations, 129 


institutionalizing expertise in industrialized countries, 127 


as knowledge-based work, 132 


needed to establish standards in a domain, 143 


needed to turn excellence into paid work, 142 


occupational control and, 131-132 


powerful motivating force of control “at a distance,” 131 


providing a normative value, 128 


providing complex, discretionary services to the public, 130 


requiring professionals to be worthy of trust, 129 
sociology of professional groups, 128 


subjecting individuals to the needs of the community, 128 


as a unique form of occupational control of work, 130 


professionalization, 129, 131 
professionals 


accumulating experience without increasing objective performance, 75 


expert role attributed to, 141 


extensively engaged in dealing with risk, 130 


many working independently without peers, 746 


most reaching a stable, average level of performance, 745 


some acquiring confidential information, 129 


“professionals profess,” 129 


professors, 221 


Programme for International Student Assessment, lessons from, 477 


Programme of International Assessment of Adult Competence data (OECD), 
114 
programs 


designed to directly develop essential probabilistic reasoning skills, 497 


functioning as “mid-range” theoretical constructs, 63 
progressive deepening, 604 
progressive temporal occlusion, 679 
projections 
creating contingency plans for avoiding or dealing with negative events, 
724 
from current events and dynamics to anticipate future events, 716 


projects, working on multiple, 296 


Prolog computing language, 89 

prolonged learning, expertise as an outcome of, 51 
promising moves, ability to recognize in chess, 820 
proof-by-contradiction, method for, 87 

proofs, LT creating some novel, 60 

propeller, amounting to an air screw, 819 
properties, using to specify relations, 99 


property, expertise as, 23 


proposal writer, case study of, 419 


propositional knowledge, depth of, 120 


propositional networks, 333 
propositions 


taking the form of two concepts linked by a statement of relationship, 178 


within a Concept Map, 176 


proprioceptive input, 568 


prosopagnosia, difficulty perceiving faces, 235 


prospect evaluation, 489, 490 


prostatectomies, completed by inexperienced practitioners, 346 
protective mechanisms, guaranteeing that older employees are more likely to 


be in stable positions, 849 


proteins, 823 


“protestant science policy,” 135 


protocol analysis 


lop) 


capturing expert thought with, 192—207, 749 
as a methodology, 195-198 
methods of, 15, 193 


as a primary tool, 207 


applications to expert performance, 203-2 


prototype user-interfaces, formative evaluation of, 187 


prototypes, 333 

prototypical cases, knowledge of, 99 

prototypical situations. See also schema(s) 
learning, 722 


proven experience, of an expert, 127 
proverbs, 644, 779 


proximal cues, 777 


“pseudo expertise,” 16, 431 
psychiatrists, showing poor decision performance, 142 


psychical objects, resisting being entirely explained, 133 


psycho-affective fidelity, 341 


psychobiography, 310, 311 


psychohistory, 310, 311 
psychological adaptations, occurring in various types of practice activities, 
653 


psychological evidence, applicable to all professional writers, 414 


psychological explanations, for skill in observational drawing, 580-586 
psychological mechanisms, producing expert performance, 771 
psychological perspectives, studies of expertise from, 59—77 


psychological processes, underpinning expert performance in sport, 653 


psychological research, based on the theory of tacit knowledge and practical 
intelligence, 770 


psychological safety, as a mediator of virtual team performance, 518 


psychological sciences, simulations in, 60 
“The Psychological Seminary of Cornell University” (Mitchell), 616 


psychological skills, maintaining expertise, 276 


psychological study, of historical figures, 310 
psychological traits, 213, 390, 564 


psychological trauma, triggering musicians’ dystonia, 566 


“psychological” triggering factors, assumed degree of, 565 


psychological uncertainty, involved in the completion of deductive rule- 


induction tasks, 485 


psychology, writing compared to writing in the humanities, 419 
psychology branch, of AI, 86 
psychology of expertise, relations to various fields, 65 


psychometric ability factors, 835 


psychometric approaches 


to assessing existing knowledge and skills of individuals, 228 


to chess skill capitalizing on the chess rating scale, 598 
predictors, 214—216 


studying the structure of expertise, 213-230 
psychometric battery, including measures of complex working memory to 


musicians, 707 


psychometric decision science studies, 486—493 


psychometric factors, performance on tasks described by, 7 
psychometric intelligence, reliable impact of, 836 


psychometric intelligence research, 485, 497 


psychometric numeracy test, 477 


psychometric reliability, 214 
psychometric studies, 494 
psychometric testing, Stumpf developing, 535 


psychometric tests 
for admitting students into professional schools and academies, 10 
of intellectual abilities, 835 


psychometric tradition, considering facets of fluid intelligence, 838 


psychometrics 


affinity with historiometrics, 313 


compared to historiometrics, 310 
defined, 213 


discussion and challenges for future research, 229-230 


psychomotor adaptations, not declining inevitably with old age, 540 
psychomotor skills, relationship with SA, 729 

psychopathology, 317 

psychotherapists, 747, 753 


public interest, promoting and protecting, 131 
public performances, of classical music, 566 


public sector, “hybridized” professionalism in, 132 


published research, categories of, 315 


“pure” perceptual experience, 151 


pure perceptual processes, 152 


pure tones, 559 


purposeful practice. See also deliberate practice 
criteria for, 398 
in domains characterized by complex indeterminate causation, 400-403 


effects of more pronounced in the older chess players, 843 


in entrepreneurship, 396—397, 398 


experts engaging in, 75 


focused on continual improvement, 389 


key role for expertise, 837 


weak interaction between age and, 846 


pushing performance beyond normal, methods of, 758 
putamen, 561 

putting, on a practice green, 205 

Pythagoras, 134 


Q-morphisms, 721 


qualifications, of an expert, 127 


qualitative, retrospective recall approach, 262 
qualitative differences, in expertise, 36 


qualitative interviews, 261—262 


quality 
of training hours, 276 


of vocabulary knowledge, 639-640 


quality control, through internalized values, 137 


“quality” practice, trying to ascertain markers of, 264 


quantitative analyses, features of in historiometrics, 310 


quantitative dimension, of “historical,” 310 


quantitative reasoning, 485 
quantitative skills, 485, 497 
quantitative social indicator, used for social decision-making subject to 


corruption, 432 


quasi-longitudinal methods, of collecting data, 266 
questionnaires 


eliciting practice-related information, 262 


quantitatively oriented, retrospective, 261 


studying self-regulation of learning, 277 
Quételet, Adolphe, 311 


race car drivers, 221, 358 


racquetball players, 685 
radar, Wilkins’s invention of, 817, 828-82 


co) 


radio waves, 818 


radiologists 


detecting relevant aspects of perceptual stimuli, 233 


diagnosing normal or abnormal, 332 


experienced, reaching a stable accuracy level, 204 


expert, needing only a few fixations, 234 
looking for inserted artificial modifications, 237 
only the less skilled engaged the LOC, 236 


showing good decision performance, 142 


railroad crossings, 361, 362 
Rajan, 199 

Ramanujan, Srinivasa, 624 
RAND Corporation, 60 


random coefficient modeling, 217 


random generate and test, 796 


random mutation, 796 
random positions 

in chess, 601 

recall of, 600 


randomization manipulation, 242 


randomness as genesis principle, 796 


rap musician, practice for, 544 


rapid categorization, of information, 718 


rapid chess, small decrease in play quality during, 702 


rapid recognition, leading to incorrect or inferior action, 74 


rapid retrieval, from long-term working memory, 420 


rapid rise in ability, during teenage and young adult years, 847 


rates of improvements (ROI), 279 
rating scale, for chess, 597 


ratings, limited in inherent subjectivity, 590 


rational decision making, goal of, 479 


rational individual decision making, 400 


rational thinking, 482 
rationality, 478 
Rationality and Intelligence (Baron), 482 


Raven’s Advanced Progressive Matrices, 490 
RAWES heuristic, 457 

reaction time delays, within “real world” tasks, 678 
reaction times (RTs), 198 


reactions, replicating themselves, 292 


reactive consequences, 197 


reactivity, avoiding, 193 


reader representation, holding in working memory, 423 


readers, referring back to lexical items in a text, 635 


reader’s perspective, taking fully into account, 424 
reading 
learning by, 92 
lexical coverage required, 635—636 
reading and editing processes, as complex, 414 
reading comprehension, 493, 715 


reading skills, Matthew effect in, 218 


real performance situations, accessing, 665 


realism, 590 


realistic observational drawing, 577 


“reality testing,” needed for viable forecasting, 297 


“realization problem,” 60 
real-time interaction, 642 


“real-world” context, looking at performance in a, 305 


“real-world” creative problem-solving task, impact of constraints on, 303 


“real-world” events, cases referring to, 293 


“real-world” implications, of studies of expertise, 291 


real-world jobs, cognitively demanding, 223 


real-world problems, 85, 91 
real-world task environments, making decisions in, 65 
reasoning, 89. See also expert reasoning 

behavioral forms and temporal skills in, 74 


biases and limitations, 453 


in Carroll’s estimates of the fluid intelligence factor, 484 
“dual process” model of, 333 


evidence-based instructional approaches to teaching, 338 


in expertise, 73 


factors explaining fluid intelligence, 485 
forward from available data as well as backward from overall goals, 99 


heuristic including, 821 


intertwined with knowledge, 67 
mechanisms occupying a critical developmental role in learning, 337 
models for seven proficient forecasters, 184 


proceeding by two very different strategies, 333 


progressing through a number of transient stages, 337 


role of emotion in, 347 
uncertain knowledge and uncertain data in, 91 
with uncertainty, 91 
recall 
limitations to, 264 
measure of error, 683 


paradigm, 683 


performance, 683, 705 


structured by major goal-related sequences, 69 
task pattern, 681 
receivers, remaining vigilant against deception, 44 


receptive functions, musical training plastically altering, 557 


receptive knowledge, 642 
reciprocal relationship, between practical intelligence and tacit knowledge, 
774 


reciprocal teaching and learning, 113 


recognition 


not an option in solving unfamiliar complex problems, 205 


of a particular type of event or situation, 722 


speed of for trained tasks, 74 


recognition-based behavior, 74 

recognition-based components, of skill, 73 
recognition-based problem-solving, by experts, 73 
recognition/metacognition (R/M), 457 
Recognition-Primed Decision (RPD) Model, 166, 168, 18 
recognition-primed decision making (RPD), 457 


ee) 


“recognition-primed” decisions, 386 


records, bearing on the performance of a single expert, 291 
Reducing uncertainty, in RAWES, 457 


reduction 


abstraction as, 133 


in weekly serious chess study, 763 


redundant information, processing, 804 
redundant learning activities, 805 


Reference Class and Class-Inclusion Neglect, 490 


reference image, superimposing a standardized grid on, 588 


referent, associating a new form to, 642 
refinement, from many repetitions, 424 
reflection 

mediating expertise, 72—73 

as a metacognitive skill, 277 
reflective conversation, 380 
reflective thought, 783 


reflexive level, teams sharing cognition at, 513 
reframing, 34 
regional transfer, 821 


register, knowledge of a word’s, 643 


regression analyses, 279 


regression equations, depicting the validity of various cues, 777 
regression to the mean, 216 
regulation, value in learning, 73 


“rehearsal,” of instructional routines, 442 


rehearsal strategy, 240 
reiterative thinking inside the box, 832 
rejection of the past, breakthrough thinking calling for, 818 


“relational agency,” 141 


relational distances, between players, 684 


“relational expertise,” 141 


relationships, between students and content, 432 


relative experts, making use of, 141 


relative motions, between players, 684 
relevant knowledge, growing, 92 


relevant structure, improved encoding of, 620 


reliability 
assessing, 214 
of data, 263—264 


estimating for a measure, 214 


implicit in experimental research, 229 


of procedures, 294 
psychometric, 214—215 


referring to the “consistency” or “repeatability” of measures, 263 


systematic review and evaluation of methods, 266 
reliance on everyday experiences, limitations of, 116 
Rembrandt, 581 
remote associates 


in creative thinking, 813-820 


perspective, examples of, 813 
perspective on creative thinking, 832 
utilization of, 831 

Remote Associates Test (RAT), 819 


remote associations 


detecting among unrelated words, 416 


importance in creative thinking, 814 


modern perspectives on, 815—820 


notion of, 813 


remote collaboration, cognitively demanding, 459 


remote-associates view, 812 


as basically misdirected, 832 


on creative thinking, 813 
on novel ideas, 812 


re-examination of case studies, 828-830 


support available for, 831 


remotely associated ideas, coming together, 814 
“remoteness,” scale of for measuring transfer of expertise, 821 


repeated (multivariate) analyses, of (co)variance, 279, 280 


repeated practice, in solving problems, 804 


repertory grids, identifying tacit knowledge a priori, 785 


repetitive movements, in over-trained monkeys, 567 


repetitive practice, compared with deliberate practice, 445 


repetitive routines, experts acquiring differences from novices, 4 
repetitive tapping rate, showed typical age-related decline, 844 
reporting verbally, on particular cues, 679 

representation method, capturing expertise, 99 


representational depictions, 577—578 


representational momentum, 684 
representational perspective, 343 


“representational redescription,” 780 


representations 
acquiring systems of, 756 
assisting in design cognition, 379-381 


developing the necessary, 760 


differences associated with templates, 610 


functionality of expert, 71—72 
higher level, 708 
of some physical system, 720 


representative situations 


advantage in memory recall, 697 


identifying from the domain, 203 


from real chess games, 699 


representative understanding. See also understanding 
promoting across evolving domains and conflicts, 498 
rather than rational optimization, 476 


sophisticated, affectively charged, 483 


reproductive thinking, 817 


research, comparing experts and novices in a given domain, 68 


research designs, of historiometric studies, 315-316 


research methods, for studies of design, 373 


research paradigms, evaluating anticipation and decision-making, 664 


“research participants,” departing significantly from the norm, 322 
research procedures, importance of appropriate, 135 


research strategies, changed to identifying mechanisms, 849 


research subjects, assembling, 313 


research traditions, risks associated with aligning different, 106 


researchers, having greater data collection power, 266 
residents or stakeholders, focusing on the local knowledge of, 138 
Resolution Method, 87 


resource management deficit, characterizing non-experts, 358 


resource requirements, cases appraised with respect to, 297 


resources 


acquiring, 398 


expert teams optimizing, 509 


optimizing other life domains, 849 


responding, slowing down while writing, 417 


response bias, change in with experience, 734 


response hierarchies, 832 


response planning, advance information for, 678 


response times, in solving algebra problems, 620 


“responsible research and innovation,” 138 
restructuring 


based on failure, 822 


as the basis for creativity, 817-819 

central in the Gestalt psychologists’ discussion of insight in problem- 
solving, 813 

of situations, 817, 823 


restructuring and remote associations, as separate mechanisms, 814 


results, experts consistently producing, 40 
retention and transfer of skills, almost the reverse conditions promoting, 668 
retinal image, arising through an infinite number of possible configurations of 


real-world objects, 584 


retraining, requiring several years to succeed, 568 


retrieval cues, kept in STM, 701 


retrievers and task performance, 219 


retrieving from memory and decomposition stage, 622 


retrospective approaches, collecting and assessing practice history data, 
260-261 
retrospective interview technique, 392 


retrospective practice histories, 265 


retrospective reports, 198 


retrospective study design, 276 


retrospective verbal explanations, 195 


retrospective verbal reports, 664 
retrosplenial cortex (RSC), 240, 242, 243, 244 


reviewing, text, 414 


revision, of writing, 415, 423 


Revisiting theme, 784 


rhetorical “problem space,” 414 


rhetorical style, required in a given domain, 419 


rhymes, used by artisans in Early Imperial China, 116 


rhythmic gymnastics, deliberate practice for, 757 


“RIASEC” model, Realistic, Investigative, Artistic, Social, Enterprising, and 
Conventional, 224 

“rich getting richer,” 218 

rich learning, promoting, 117 

Rigged up Perception-Action Systems (RUPAS), 155 

right fronto-parietal infarct, 629 


right fusiform gyrus, 239 
right hemisphere, 555 


involved in abacus calculations, 629 


solving problems through insight, 819 
right medial frontal and parahippocampal gyri, 628 
right motor hand area, piano training and, 560 


right primary auditory cortex, 554, 559 


right putamen, size of correlated with the age at which music training began, 
962 
right-handedness, 415 


right-hemisphere involvement, in a range of tasks, 628 


risk 
evaluating and understanding, 478 


exerting a direct effect on affective responses, 496 


involving, 479 


as a preoccupation of economists, 390 
in selecting a chess move, 606 
tolerance, 357 

understanding, 498-499 


risk literacy, 478, 492, 495, 49 


risk management, issues of, 467 


ep) 


risk perception, consistency in, 486, 487 


risk-taking, between groups of countries, 606 


Riviera, Diego, 827 
Robie house, Frank Lloyd Wright Prairie House, 826 


robotic patient simulators, 332 


Rod-and-Frame illusion, 583 


roles. See expert role(s) 


room layout, reorganized, 174 


root contacts, during dental surgery, 346 


rough draft, preparing, 418 


routine experts, implementing set routines, 436 
routine processing, versus controlled processing, 73—74 
routines 


considering a large number of, 440 


developing adaptive knowledge of, 442 
experts using, 455 

Royal Academy of Music, 9 

Royal Designers for Industry, 373 


royal or princely courts, 136 

Royal Society in London, 136 

Royal Society of Arts (RSA), 373 

RPD. See recognition-primed decision making (RPD) 
RSC (retrosplenial cortex), 240, 242, 243, 244 

rugby playmakers, training programs for, 458 

rule, 87 

“rule induction” tasks, 485 


rule-based architecture, common, 74 

rule-based paradigm, experiments with, 96 

rule-based representation, discovering new rules of mass spectromerty, 88 
rule-based systems, 87 

rule-induction, in fluid intelligence tests, 493 

rules, as essentially incomplete, 24 


runners 


aging effects on expert, 842 
elite endurance, 73 

marathon- or half-marathon, 842 
measuring speed of, 214 
middle-distance, 759 


running economy, on a treadmill, 749 
running or walking, helping some writers think through problems, 422 
Ruskin, John, 581 


SA. See situation awareness (SA) 
SABIC, configuring a color formula, 97 


saccades, 685 


safe driving, among enlisted Army personnel, 779 
Safe Speed Knowledge Test, 779 

SAGAT mean scores, by experience level, 731 
sales pitch, 399 


salespersons, studies of expert, 402 


sample membership, assigning, 313 


samples, recruitment of large, 261 


sampling 


according to eminence, 314 


across different age groups, 266 
procedures, 313-314 
satisfactory level, 753 
SAT-M (Scholastic Aptitude Test — Mathematics), 616, 627 
SAT-V (Verbal), 627 
scaffolds, 552, 566 


scale playing, preparation required, 567 


scale-playing task, pianists completed, 562 


scan patterns, 719, 720 


“scanners,” strategy of, 219 


scenarios 
direct and indirect for hazards, 363 


experiencing through the eyes of the experts, 460 


Schadenfreude, 642 

schema(s), 234 
structure of, 99 
uses of, 722 


schema-based theories, 106 


schematic illustration, 756 


scholar-officials, in rising empires of antiquity, 127 


scholars, from diverse domains, 772 
scholars’ guild, established in the 12th and 13th centuries, 6 


school system, insufficiency of the traditional, 76 


school teaching, observing elementary and secondary, 437 


school-based writing assignments, during adolescent years, 424 


“schooled societies,” overlooking or minimizing the educational worth of 
experiences, 108 
schooling 


as a Cultural invention, 435 


privileging in the development of expertise, 115 


schools 


boosting performance of, 783 


developed to prepare workers, 435 


helping students acquire skills and mechanisms for proficient performance, 
76 
Schumann, Robert, 224 


science 


changing relationship with society, 138 
constituting the core epistemic social (sub-)system within modern 


societies, 136 


as the core accepted reference system for knowledge in current societies, 


134 


equating perceptions of control with the ability to predict, 395 


function of, 137 


1 


historical overview, 134—1 


incorporation into everyday practices, 22 
institutionalized in the Western world, 135-136 
linked to the idea of progress, 135 


linking signs and symptoms to diseases, 339 


as the main reference system for knowledge, 127 


as a marginal activity within early universities, 136 


operating conditions of today’s, 136-138 


as a profession, 137 


referring to technologies and systematized knowledge, 143 
as a social system, 137 


within communities of people sharing scientific interest, 135 


“science as a vocation” (Weber), 137 


science/math trait complex, 226, 229 
The Sciences of the Artificial (Simon), 64 


science—society cooperation, 138 


scientific contribution of a scientist, 221 


scientific creators, growing up in stable and conventional homes, 317 
scientific cultures, required by professional science, 137 


scientific enterprise, clear vision of the purpose of, 135 


scientific leaders, assessing the performance of, 295 
scientific management, development of, 296 
scientific method, 135 


scientific misconduct, 137 


scientific paradigm, Thomas Kuhn’s idea of, 23 


scientific research, on expertise, 233 


scientific worldview, of Aristotle, 134 


scientific writers, work sessions, 422 


scientists 
area expertise of, 301 
developing and extending science as a knowledge base, 134 


dimensions of, 301 


interviews of peer nominated eminent, 13 
leaving extensive records, 300 
needing distinct educational experiences, 318 


outstanding, best work at around 35 years of age, 751 


testing aspiring performers during development of expertise, 14 
“thinking like scientists,” 33 
Scrabble game, 203 


screening procedures, for artist participants, 589 


script concordance tests, 776—777 
scripts, 254 
developing for particular diseases, 338 


tying to schema, 722 


Sscript-scenario instrument, 392 
Scripture, E. W., 616 


sea transport, expertise in, 356 


search and pattern recognition, dissociating in chess, 603 


search heuristics, 88 

search mechanism, “short circuiting,” 823 

search pattern, determining the most effective, 661 
SEARCH probabilistic model, 602, 605 


search process, evaluating moves in chess, 598 


search space, size of, 88 
search-based approaches, not efficient as instructional methods, 805 


searches, based on failure, 822 


seasonality of birth, as a marker for chess talent, 610 


second language (L2) learners 
requiring much smaller amounts of vocabulary, 635 


use of formulaic language by, 646 


secondary auditory areas, 554 


secondary auditory cortices, 558 
secondary competencies, 42—44 


secondary knowledge, 795 


secondary market research, for entrepreneurship exercise, 406 


secondary motor areas, processing movement patterns, 555 
secondary olfactory areas (orbitofrontal cortex and hippocampus), in blind 


persons, 239 


“see the system,” learning to, 444 


“seeing as,” 380 
seeing problem, in algebra learning, 620 


“seeing that,” as reflective criticism, 380 


selection 


basing on perceptual speed and psychomotor measures, 223 


in drawing differing between artists and non-artists, 587 
selection forces, changing to social pressures, 41 


selective combination, 774, 782 


selective comparison, 774, 782 


selective encoding, 774, 775, 782. See also encoding 
9 


selective maintenance account, 83 


selective optimization, with compensation, 540 


selective search, by chess players, 603 
selective skill maintenance interpretation, 844 


selectivity, as a weakness of case studies, 830 


self-checks, introduced to more knowledgeable learners, 807 
self-concept, 222, 225 
self-control, 132 


self-designed versus coach-led practice, 656 


self-determined practice, 260 


self-driving cars, 366 
self-education, various forms of, 318 
self-efficacy, 222, 225, 277 


self-estimated performance ability, for older musicians, 540 


self-explanations 
benefits of, 339 


improving participants’ activities, 197 


self-monitoring, in expertise, 73 


self-observation, 192 


self-organization, 127 


self-purification, 139 


self-regulated and effortful activities, 392 


self-regulated learning, optimizing, 785 


self-regulation, 398 
of experts, 73 
of learning, 277, 278 


methods for writers, 422 


motivation (need) to practice and, 536 


research on, 539 
right of, 137 


Self-Regulation of Learning Self-Report Scale (SRL-SRS), 657 
self-regulatory activities, 260 


self-report measure, for TMS, 512 


self-report method, for measuring expert teams, 518 


self-report surveys, measuring teamwork in medical teams, 518 
self-reports 


avoiding pitfalls associated with traditions, 404 


of maladaptive real-world decision outcomes, 488 


semantic axes, 333 

semantic concepts, not all transferable between languages, 642 
semantic markups, inferring knowledge from, 93 

semantic memory, 61, 620 

semantic web, 93 

semi-professional work, speed-up of, 96 


senior competitions, age of transition for, 273 


sense of self (subjectivity), 120 


sense-making, 456 


cognitive process of, 172 
commencing anywhere, 461 
described, 461 


ensuring and maintaining consistency among indicators, 466 


in a hybrid ecology, 466 
proceeding in fits and starts, 461 


in real-world situations, 461 


Strategies involving, 457 
sensitive periods 
of the brain, 569 


existing in the nervous system, 551, 563 


sensitivity 
of experts to linked segment information, 680 


to timing variations, 552 


sensor based team measurement, in the field of healthcare, 519 


sensorimotor delay, 684 
sensorimotor expertise reversal effect, 803 
sensorimotor skills 


expertise reversal in training, 803—804 


explicit monitoring theory of, 802 


sensory abnormalities, driving a motor disorder, 568 


sensory acuity, changing, 151 


sensory homunculus, 556 


sensory input, altered in musician’s dystonia, 568 


sensory motor control, brain changes associated with, 566—568 


sensory perception, altered, 566, 567 


sensory retraining, in the form of tactile discrimination practice, 568 


“sensory trick” phenomenon, 567 


sensory-motor domain, 551, 560 


sensory-motor integration 


changes in, 558 


impaired, 566 


impaired role in musician’s dystonia, 567 


sensory-motor programs, early optimization of, 566 


sensory-motor skills, 552, 563 


sentences 


involving the language zone of the left hemisphere, 415 


retaining meaningful, 622 


writers generating cohesive links among, 414 


separateness, of knowledge and domains of expertise and experts, 34 
Sepsis, 458 


sequencing, 404 


sequential reasoning, 485 


sequential thinking, 515 
serial evaluation, via mental simulation, 457 


“serial-cue” approach, 338 


serotonin, increased release of, 553 


service and autonomy, making professionalism attractive, 131 


“service” orientation, of professionalism, 128 
severe time constraints, inducing anxiety, 347 


sex difference, “environmental” hypotheses and, 627 


sexing chickens, 152 


sex-linked characteristics, contributing to mathematical expertise, 627 


sexual reproduction, information reorganized during, 796 


ShadowBox® training, 182 


accelerating progression towards expertise, 184—185 
recent work on, 404 


scenario-based training approach, 460 


shamans, in pre-modern societies, 127 


shape constancy 
described, 582 
effect, 582 
errors, 589 
illusion, 581 


“shared,” meaning “distributed,” 459 


shared cognition. See also cognition 


current research on, 513-514 


forms of, 511 
pertaining to time affecting team performance, 513 
researchers considering the literature on TSM and SMM separately, 521 


of teams, 511 


shared conceptions, developing via dialogue, 403 
shared instructional objects, 445 
shared knowledge structures 
held by members of a team, 512 
of SMM, 513 
of teams, 509 
shared leadership, 509. See also leadership; team leadership 
behaviors distinct from individual-level leadership, 520 


conceptualization of, 515 

contributing to both team and individual creativity, 509 
contributing to knowledge sharing, 509 

defining, 520 

distributed among the team, 520 


integral in virtual project teams, 514 


lack of construct clarity, 514 


measured via behavioral, affective, and cognitive constructs, 515 


meta-analysis on, 514, 515 

multiple forms of, 515 

research, 520 

role of, 509 

separating expert teams from others, 515 


structures, 514 
shared mental models (SMM) 


based on deliberate and conscious processes, 513 


categorized into task and team related mental models, 513 


conceptualizations of, 459, 513 

content of, 513, 521 

contributing to enhanced team processes and team performance, 513 
defined, 512 

distinguishing from TMS, 512-513 


encompassing knowledge shared among all team members, 512 


encompassing task content as directly related to performance, 521 


expert teams holding, 509 


important role on the performance of virtual teams, 518 


measurement of, 512 


representing team members’ knowledge, 458 


types of, 513 
shared professional identity, 130 


shared temporal cognition, mitigating the negative impact of diversity, 513 


shared values, of scientists, 137 


shared vocabulary, 762 
Shark Tank (TV show), 399 
Shaw, Clifford, 59 
Shepard illusion, 581 
shogi, 243, 244 
shopping cart 

IDEO creating a new, 824 


innovative aspects of, 824 
short term results, practices yielding impressive, 431 


shorter-term predictions, likely to be much more accurate, 220 


short-term memory (STM). See also working memory 


activating knowledge from long-term memory (LTM), 696 


amount of information held in, 68 
cognitive processes uniformly constrained by, 70 
constraints, 72 


decision makers circumventing attentional capacity limitations of, 483 


invariant limits on, 76 
skill not residing in differences in, 600 


shot sequencing, access to varying levels of, 664 


sighted participants, brain activity of, 238 
sight-reading, 555 


improving with overall musical ability, 543 


influence on working memory, 707 


of music notes engaging a number of areas of the brain, 237 


by pianists, 746 
playing music by, 707 


predictors of performance, 707 


signal value, of expertise, 45 
signaled traits, 46 
signalers, dissembling high quality traits, 44 
signaling theory, 44 
signals 
development of, 44 


types of in humans, 519 


significance, testing, 280 


significant samples, 313 
similarity 
among team members, 515 


moderating effect of versus accuracy of the team’s SMM, 521 


of SMM measurements, 512 


Simon, Herbert, 59, 61 
doing psychology, 61 


Grandmasters of chess viewed as intellectual prodigies, 696 


start of modern laboratory research on expertise attributed to, 697 


Simon—Chase theory of expertise 
approaches drawing on, 12 
limits and criticisms of, 697—699 

simple check task, 241 


simple games, no such thing as expert performance, 45 


simple shapes, detecting embedded, 580 


simplex-like effect, 221 

simulation(s), 341-342 
described, 120 
exercises constructed from CTA data, 460 
fidelity, 343 


increasing use of, 340 


method for measuring expert teams, 518 


in the psychological sciences, 60 

technology supporting teaching and assessment in medical education, 332 
tools using with NDM, 465 

training, 460 


using videos and standard actor patients, 338 


simulation-based clinical education, 341 


simulation-based learning environment, 342 
simulator-based flying accuracy, 845 
simulators 


high-fidelity used in airline operations, 465 


training with, 75 


simultaneous translation, working memory during, 704—705 


singers 


amateur experienced lessons as self-actualization, 761 


classical starting later, 536 


educational traditions affecting practice duration, 538 


found to have larger vital and total lung capacities, 542 


working with an accompanist, 538 


single-case designs, 316 
Siqueiros, David Alfaro, 827 


site constraints, vis-a-vis the principle of maximizing sunlight, 302 


“situated” act, designing as a, 379 


“situated cognition,” 461 


situated domain of practice, 111 


situated expert performance, 122 


“situated learning,” 23 


“Situated learning: Bridging sociocultural and cognitive theorizing” (Billett), 
106 


situated manifestations, of occupational expertise, 110-11 


—_ 


situated practices, 111 


situation(s) 
calling for immediate action with imperfect knowledge, 453 
emphasis on mutuality in, 402 
expert reading of, 442 
familiar, 455 


not needing to be exactly like previously encountered situation, 722 


recognizing as typical, 386 
situation assessment, 179 
of experts, 456 


information most critical to accurate, 455 
most effort on, 453 
situation awareness (SA), 442, 715-71 


cognitive model of, 716 
data available, 717 
defined, 714, 715 


developing in the infantry environment, 732 


error causal factors across pilot groups, 727 
expertise and, 714—736 


factors affecting in novices and experts in a domain, 720 


forming a central and conscious task for expert drivers, 735 
gathering of as an active process, 723 
hallmarks of expert, 726 


importance for army operations, 730 


increasing with automation, 735 


integral to many domains, 735 


Level 1—2-3 progression, 718 
Level 1-Perception, 715, 717, 725, 727 
Level 2/3, 718 


Level 2-Comprehension, 715 


Level 3-Projection, 716 


low, associated with cognitive automaticity, 723 


problems for new platoon leaders, 732 


qualitatively different depending the level of experience, 731 
ratings for pilots across four groups, 729 


requiring “awareness” of information by definition, 723 


role of expertise in, 719-726 
skills required, 723-724 


situation model development, in skilled reading comprehension, 482 
situation or problem, reaching an understanding of, 456 


situation specific context, 665 


situation typicality, 457 


situational and structural effects, 347 
“situational awareness,” 71 


situational characteristics, demanding tacit knowledge, 784 


situational/event probabilities, in sport, 662 


situational-judgment testing format, 776 


situationally derived goals, responding effectively to, 110 


six-year-old children, effects of 15 months of piano training, 559 


size and shape constancy errors, 583 


size constancy 


artists and non-artists showing effects, 582 


comparing with visual selection advantages, 589 
described, 582 


effects smaller among experienced artists, 589 


size matching task, depth cue condition of, 587 
sketching 


assisting cognition in design thinking, 380 


enabling exploration of the problem space, 381 

purposes of, 380 

tied in closely with features of design cognition, 380 
skill(s) 


acquired over time, 50 


adapting effectively to new contexts, 343 


affecting how progressive deepening is carried out, 604 


altering or circumventing the processing limits of attention and working 


memory, /6 
as being first cognitive, then associative, and later autonomous, 540 


constant “stretching” through challenging cases, 464 


deteriorating unless actively maintained, 839 


developing through life, 540 
differences of experts, 481 
in drawing, 583 


highlighting the specificity of, 340 


knowledge mediating in chess, 599 


learned explicitly less likely to be forgotten and more robust under stress, 
688 


maintaining, 845—846 


measures of retention and transfer, 667 
needing to be explicitly taught and consciously practiced, 795 


obsolescence of becoming a risk, 849 


performed in dynamic and unpredictable contexts, 343 


in playing chess transferring to other domains, 607 
processes of formation, 115 


of professional writers, 422 


progressions, 266 


referring to acquired types of knowledge, skills, abilities, and related 
Capacities, 477 


relevant to obtaining critical information, 723 


required for expert teachers, 441—442 
role in life history development, 53 
training literature, 690 


training of critical, 464 


skill acquisition 


in chess requiring a considerable investment, 608 
in early and later “starters,” 569 


experiment with a simplified air traffic controller (ATC) task, 218 


five-stage general model of, 385 


involving the adaptation of pre-existing mechanisms, 591 
models of as general cognitive architectures, 60 
phases of, 115 
specific adaptations in cab-drivers, 841 
for writers, 422—426 
skill components, 845, 846 
skill development, 42, 51, 262, 540 
skill level 


continuing to improve during adulthood and stabilizing, 569 


correlated moderately with the cumulative amount of individual practice in 
chess, 609 


not correlating with handedness, 610 


skill process changes, in animals, 54 

skill-by-structure interaction, 541 

skilled artists, guided by the veridical two-dimensional appearance of the 
models, 581 

skilled athletes 


developing high-level knowledge structures in memory, 662 


intercepting fast-moving targets, 684 


recalling player positions, 661 
skilled batters, gaze behaviors of, 662 
skilled chess players 


having superior memory of chess positions, 697 


rapidly and accurate identifying a threat, 603 


rapidly selecting chess moves, 201 


tending to be inflexible, 74 
using knowledge about chess configurations, 598 
Skilled Decision Theory, 482—483 
skilled decision-makers, 481, 482 
skilled decision making, 479, 497. See also decision-making 


skilled encoding in LTM, key constraint for, 71 

skilled individuals, looking further ahead in the text or music score, 706 
Skilled Memory Theory, 482 

skilled object perception, in chess, 242 


skilled perception, appearing resistant to age-related declines in handball 


goalkeepers, 274 


skilled performance, developing, 115 
skilled performers, facilitating superior performance, 686 
skilled players 


ability to use higher-order cognitive information, 662 


capacity to make perceptual judgments using information picked up solely 
by peripheral vision, 686 


more accurate at sports judgement, 662 


needed a larger area around fixation to detect changes, 599 
superior anticipatory skills, 687 
using different anticipation search strategies, 664 


skilled radiologists, example of, 233 


skilled soccer players 
group of with little to no basketball experience, 684 


more often generated the best options (as assessed by coaches), 706 


reported more relevant events and actions away from the ball, 207 


skilled typing, research on, 706 


skilled volleyball players, predictions of, 686 
skillful knowledge, 165 
skill-related tasks, aging and, 840 


skill-sustaining deliberate practice activities, constrained by advancing age, 
846 

Skinner, B. F., 63 

slogans, 644 


slower learners 
acquiring skills necessary for an expert level, 218 
catching up over time, 222 
SMA (supplementary motor area), 555, 556 
“small theories,” 440, 447 
SMG (bilateral supramarginal gyri), 241, 243 
SMM. See shared mental models (SMM) 


smokejumpers, 830 


snooker players, 205 


soccer players 
asked to state their recommended action, 705 


attainment of a professional contract related to early types of practice, 263 


comparing practice history profiles, 655 
deciding who is the better player, 273 
“describing (aloud) actions taking place on the field,” 207 


evaluating midfielders, 747 


film sequences from the perspective of a central defender, 664 


goalkeepers, 678 
strongest predictor of decision making performance, 688 
tests at particular ages correlated with attaining highest levels as adult 


players, 752 


training aerobic fitness, 760 


verbal reports when moving freely in front of a life-size video screen, 206 
youth, 263, 277 


social (sub-)systems, interacting with one another, 137 


social activity, sport as, 655 
social actors, constructing and contesting expert status, 22 


social and cultural world, shaping human activities, 121 


social and technological outcomes, putting indigenous or minority 
communities at risk, 22 
social complexity, limiting the development of expertise, 46 


social conception, of the expert, 401 


social construction, expertise as a, 50—51 
social constructivist approach, 22 
social demands, coping with, 42 


social division of labor, 141 


social dynamics, anticipating and coping with, 41 


social elite, scientists and professional experts as, 138 
social embedding, in an expert community, 23 


social emotions, evoked by music, 554 


social exchange relationships, 299 


social exchange theory perspective, studying shared leadership, 514 
social fluency, sociological model of expertise as, 27 
social groups 

collaborations between, 25 

defining the culture of, 27 

expertise as a property of, 28 

mobilizing cultural and epistemic resources, 22 


social hierarchy, moving up, 41 


social innovation 
case studies, 296—297 
described, 296 


experimental studies of, 297—298 


problems, 296 
social integration, fostering unit-level, 516 


social interaction, 645 


social judgment analysis, identifying tacit knowledge a priori, 785 


social learning, 781 

social media, using in the L2, 648 
social nature, of designing, 374 
Social Network Analysis, 106 


social networks, building effective, 398 
social norm recognition, 486, 487 


social phenomena, applying statistics and probability theory to, 311 


social practices, participation in, 23 
social pressures, 41 
social scientists 
justifying research, 26 
understanding fieldwork setting, 27 
social signaling, expertise and, 44—46 
social stressors, within groups, 467 


social system, science as, 136—137 


social systems (teams), categorized by degree of complexity, 511 
“Social” trait complex, 226 

social use of expertise, 141 

social voting criteria, for expertise, 50 


sociality, 402 


socialization 
acquisition of expertise through, 21-28 
as the foundation of expertise, 21 
linguistic and physical aspects of, 28 
socially relevant research tasks, 138 


societal discussion, about the changing nature of work, 100 


societal expectations 


on literacy, 436 


for occupations, 109 


societal laws, recognizing the value of skilled or expert animals, 50 
societies 
marked by a division of labor and specialization, 45 
modern functionally differentiated, 136 
society 
developing experts in content knowledge who use their knowledge to bad 
ends, 780 
encompassing cultural and institutional provisions, 143 
socio-contextual variables, influencing case reporting, 295 
sociocultural context, expertise of the highest order likely to appear in a 


particular, 318 


socio-cultural perspective, emphasizing social negotiations, 121 


socio-cultural popularity, of a sport, 667 


socio-economic conditions, of a young musician’s family, 536 


sociological interest, in the enactment of expertise, 22 


sociological point of view, expertise as professionalized, 127 


sociological/philosophical perspective, on expertise, 21—28 


sociology of professional groups, historical account of, 128—12 


socio-technical work systems, working within, 187 


Socrates, on expertise, 5 

soft modularity, 42, 46 

soft modules, top-down modification of, 42 

software design, deliberate practice and, 75 

software system designers, protocol analysis studies of, 383 
software technologies, 172 


software tools, facility with, 178 


soi-disant professional artists, recruited from the community, 589 


solitary activities, allowing for focused practice, 758 
solitary practice, 536, 759 


solution alternatives, rating, 776 


solution concepts, generating a range of alternative, 377 


solution conjectures 
by designers, 377-378 


exploring and understanding problem formulation, 378 


solution ideas and concepts, attachment to early, 382 
solution precedents, store of knowledge of, 381 


somatosensory areas, connection with nominally visual areas, 238 


somatosensory cortex, 567 


somatosensory input, changing, 568 


somatosensory perception, refined, 557 


somatosensory representation, of the left fifth digit in string players, 558 


“somatotopic” order, 556 


songs, 644 


sound localization, 557 


sounds, increased sensitivity to, 559 


sources, 293 


South African government, decision not to use AZT, 22 


sovereign, acceptance of the divine right of, 131 


Space, Managing within a map, 178 


space crew teams, 516 


spatial ability 
higher in more skilled crafters in middle and older age groups, 847 
robust age-related decline in, 845 


spatial “acalculia,” 623 


Spatial experts, 239 


spatial learning strategies, intensive usage of, 841 
spatial occlusion, 679 


spatiotemporal control, 561 


“special interactional experts,” 26 


special populations, continued focus on, 592 


specialist expertise, types of, 25 


specialists, generating more complete explanations, 205 


specialization, in TMS, 512 


specialized knowledge 
resting on a base of everyday knowledge, 94 
as the wellspring of high levels of performance, 85 
specialized roles, dividing into, 41 
specific human capital, 391 
“specification problem,” 60 


specificity of practice, in the surgical domain, 343, 344 


spectral aspects, musicians oriented toward, 543 
speed of most types of perceptual-cognitive-motor performance, undergoing 
age-related declines, 837 


speeded performance, ubiquity of negative age-effects in, 838 


spelling, mastering the mechanics of, 422 


spoken discourse, tending to be lexically less dense, 635 
spoken form of a word, 640, 642 


spoken or written discourse, words appropriate for, 643 


sport(s) 


accumulated hours of different practice activities, 762 
age for top performance in, 321 


application of the RPD framework to the domain of, 457 


changed standards in, 543 


deliberate practice in, 75 


evolving with higher standards of expertise, 274 


expertise in, 653-669 
expert-performance approach applied to, 205-206 
fastball, 660 


with fewer predictor variables, 666 


as inherently complex and multifaceted, 666 


motor expertise and, 65 
NDM tactics shown to improve performance in, 460 
performance happening rarely, 445 


rare to play at top form beyond the fourth decade, 847 


studying skilled anticipation in, 690 


task specificity limitation to a domain, 67 
tasks, 677 


teams playing matches against other teams, 747 


working memory and, 705—706 


sport expertise development, studies of, 265 


sport expertise literature, 681 


sport expertise studies, predictive ability of, 266 


sport performance, profiling, 258 


sporting domain, use of questionnaires, 261 


sporting play, 264 
sporting talent, predicting, 666—667 


sport-practice activities, rated some high for enjoyment and relevance to 
improving performance, 655 
Sports Car Club of America, 358 


sports career, 276 


sports science, as an academic field, 653 


sport-specific stimuli and tasks, using more, 662 


sport-specific training activities, importance of engagement in, 659 


sport-specific training history information, athlete recall of, 264 
sprinters, objective running times of, 50 
spuriousness, source of, 317 


squares or circles, viewing stimuli containing, 586 


“squid-like” concept, from an analogy in the designer’s mind, 374 


“stabilizing factors,” in the evolution of science, 137 
stable traits, 213 


staff positions, consolidating, 174 


stage and phase models, for acquisition of a new skill, 539 


stage model, of development from novice to expert, 36 
stages of development, for future performers, 77 
stakeholders 


engaged in the activity of co-creation, 403 


helping shape the venture, 400 


perspectives, 169 
relationships, 401 


setting their own terms, 400 


standard cognitive paradigm, 165 


standardized game situations, presentation using videos or scripted situations, 
206 

standardized instructions, 62 

standardized measures of performance, 37 

standardized methodologies, promoting validity and reliability of 
measurement, 265 


standardized patients, 332 


standardized performance measures, 37 
standardized tests, 296, 438 
standards 


experts establishing, 143 


of logic, probability, and statistics, 479 


used by professionals, 133 
Starck, Philippe, 374 


“start” and “stop” points, Flexecution model not assuming fixed, 462 


starting age 
playing an important role in chess, 609 
for practicing the violin, 654 


for reaching an international level of achievement, 751 


starting or halting points, for complex causation, 462 


Starting points, enabled designers to limit the problem, 375 
“startle effect,” 464 


Startup environment, repeated practice of The Ask as an inevitable feature of, 
299 
Start-ups, bootstrapped on budgets suiting every wallet, 390 
states 
described in terms of systems of representations and their interconnections, 
756 


evaluating options for desired, 180 


involved in the training of expert performers, 9 


in a physical system, 178 


representing temporary characteristics, 213 


statistic controls, helping avoid the intrusion of spurious associations, 316 
Statistical analyses 

answering the research question, 278 

choice for, 272 

in longitudinal studies, 278—280 


most people not computing, 479 


Statistical learning, 92, 93 
Statistical methods 
assessing probabilities, 88 
for perceptual and motor tasks, 93 


utilized to assess reliability and validity, 264 


Statistical modeling, rarely undertaken, 667 


Statistical numeracy, 488. See also numeracy 
as an essential component of general decision making skill, 483 


link between general decision making skill and, 489 


mediated any connection between fluid intelligence and decision making 
skill, 492 
predicting decision making skill and risk literacy, 478 


predicting general decision making skill, 479, 492 


predicting skilled decision making, 498 
as a robust predictor of numerical and non-numerical decisions, 478 
working professionals having relatively low levels, 494 

Statistical numeracy tests 


explaining 33% of the total decision making skill variance, 489 


representative judgment and decision making tasks, 478 

as robust predictors, 494 

tending to be the strongest single predictors of general decision making 
skill, 476 


statistical techniques, suitable for the analysis of correlational data, 322 


Statistical theory, 485 


“staying there” stage, with maintenance, 540 


steep associative hierarchy, 814 


steering into lane task, drivers’ gaze patterns not varying as a function of 


expertise, 362 


STeLLa: Science Teachers Learning from Lesson Analysis, 448 
STEP procedure, 457 

stereo vision, among accomplished artists, 581 

Sternberg, Robert J., 772 

stimulus and response generalization, of the learning theorists, 820 


STM. See short-term memory (STM) 


stored knowledge structures, 234 


stored movements, in LTM, 234 
stories 

drafting by hand, 416 

of lived cases, 175 


strabismus, higher rates of, 581 


strategic flexibility, of experienced artists, 587 


strategic knowledge, 90 
strategic positions, in chess, 604 


strategic procedures, required for occupational competence, 108 


strategies 


bottom-up and top-down modes of perception as, 587 


considering a large number of, 440 
experts using to fine-tune skills to dynamically changing conditions, 459 


implementing to create learning opportunities, 440 


individuals using to make superior decisions, 481 


range of exploration of possible, 436 
shifting when faced with high uncertainty or unmet expectancies, 456 


teachers deciding which to pursue, 442 


testing of alternative, 447 


used by experts as more flexible, 358 


Strategies research category, for transportation, 357 


stratification approach, 658 


strengths and weaknesses, pattern of, 228 


stress 
as a cause of diagnostic errors, 347 


promoting dysfunctional motor memory formation, 566 


stress hormones (fear of failure), 553 


stressors 
effects on SA, 719 


triggering a deterioration of motor control, 563 


string players, left hand movements, 542, 561, 841 


strong chess players, 601, 603, 604. See also expert chess players 

strong methods, dependent on knowledge of the problem-solving area, 62 
Stroop-like interference task, 599 

Structural Adaptation Theory (SAT), 511 


structural brain changes, 235 


in cognitive expertise, 244—245 


in motor expertise, 248 


in perceptual expertise, 239 


structural brain differences, reported in musicians, 561 


structural consequences, of randomness as genesis principle, 796 


structural differences, between musicians and non-musicians, 560 


structural equation modeling, 279 


structural process model, 495 
structural supports, 517 
structure 


of expert performance, 709 


extracting from the world, 442 
in interactive learning environments, 799 


structured data collection form, 260 


structured objects, 99 
structured patterns 
in basketball, 682 
experts applying, 683 


structured plan, 383 


structured practice activities, 264, 760 


structured retrospective interviews, in domains outside of sport, 262 


student achievement, identifying experts based on, 437—439 


students 
attention to individual, 443 
coming up with solution methods in Japan, 435 


cooperation of, 434 


diagnosing clinical cases, 339 


novice superior on a near transfer post-test, 798 
observation and analysis of thinking and learning, 447 


outcomes, 432 


participating in the PIFS program, 783—78 


with poor knowledge of spreadsheets, 800 
relationship with early teachers or coaches, 442 


remembering steps used by the teacher, 434 


time spending actively engaged in learning, 433 


US uncomfortable with the experience of confusion, 435 
writing samples, 437 
studies, identifying natural and experimental variations in practice behavior, 
760 
Studies of Expertise and Experience (SEE), 23 
study methods, choice of, 272, 277-285 


styles, prewriting strategies expressed as, 418 


subcortical (brainstem) response delays, for auditory stimulation, 842 


subfields, understanding the discourse of, 26 
sub-goals, 62, 114 
subject matter experts, content validity established through, 215 


subjective accuracy ratings, by independent judges, 590 


subjective assessments, based on surveys of scholars and other experts, 314 
subjective criteria, 295 


subjective data, 258 


subjective expected utility theory, 479 


subjective ratings, 590 


subjectivity, of qualitative interviews, 262 


subject-matter expert (“SME”), debriefing, 85 
subjects, asked to “think aloud,” 193 


submarine technicians, NDM training of, 460 


sub-skills, in different, though overlapping brain networks, 552 
sub-tasks, 114 


subtitles, as a useful source of vocabulary learning, 648 


success of actions, depending on the level of performance of the opposing 
players, 748 


successful maintenance, constraints on, 846 


sunk costs, resistance to, 486, 487 


superior decision making. See also decision making 
in chess, 481 
factors driving, 483 


reflecting specialized knowledge, 476 


superior individual performance of experts, 745—765 
superior longitudinal fasciculus, 245 
superior memory. See also memory 


accounting for, 200 


acquisition of, 199 


associated with more expertise, 705 


of chess masters, 11 


dissociated from superior diagnostic performance, 699 


of experts, 696 


relating superior expert performance, 708 
specificity of chess players,’ 703 
superior parietal lobe (SPL), 247 


superior performance. See also performance 
analyzing, 72 


attributing to domain-specific patterns or chunks, 579 


capturing on memory tasks, 700 


depending on expanded access to intermediate products, 708 


evidence of, 746 
of experts, 193 


identifying individuals with, 67 


mechanisms mediating, 745, 748-749 


superior performance approach, 105 


superior temporal gyrus, 559 

superior working memory, expert-performance approach to, 699-700 
supervision, during practice for beginning musicians, 539 
supervisors, support from, 522 


Supplementary motor area (SMA), 555, 556 


Suppressing uncertainty, in RAWES, 457 


supramarginal gyrus, 562 


supra-national mobility and connections, of professionals, 129 


surface level, of team composition, 515 


surface similarities, leading to predicting the future, 364 


surgeons 
cognition of, 465 


experienced, at a disadvantage, 343 


expert, “slowing down” for non-routine events, 456 
expert skill, 341 


laparoscopic, 343 


maintaining public trust, 340 


studies of expert, 72 
with superior outcomes, 67 
technical procedure outcomes, 346 


surgery and medicine, mastery approaches to technical skill education in, 342 


surgery(ies) 
direct relationship between specific surgical experience and patient 
outcomes, 346 


number of, associated with improved performance, 754 


predictors of expertise in, 345—346 


task specificity limitation to a domain, 67 
technical expertise in as acquired and highly local, 342 


surgical expertise, research on the development of, 343 


surgical knot-tying, expertise in, 341 


surgical mortality rates, as a function of the number of procedures completed, 
346 


surgical performance, 200 


surgical procedures, 340, 460 


surgical repair and the handling of any emergency situations, influencing 
patient outcomes, 748 
surgical skills, acquisition of, 73 


surgical specialties, patient satisfaction ratings of, 747 


surgical teams, 747 


survey method, identifying high-performing physicians, 345 


surveys and questionnaires, researchers and designers of, 207 


sutures 
assessments of closures, 340 
outcomes related to surgical expertise, 341 


practicing in the microsurgery environment, 342 


suturing 
proficiency, 343 
simulator, 341 


sweeps, story re-telling cycling through, 176 


swimmers, age-related effects delayed in active, 842 
switching levels, cost of for artists, 586 
symbol manipulation 

defining efficient techniques, 86 


in mathematics, 88 


symbolic and conceptual knowledge, in technologies, 117 


symbolic knowledge, about an entity, 99 
symbolic programming languages, 63 


symbolic representation, using words, 800 


symbolic structure, of equations, 620 


symbols, superior memory capacity for, 199 


“symbols and symbol structures,” processing, 59 


symptoms, understanding mechanisms of, 339 


synapses, 553 


synaptic connections, strengthening of, 553 


synchronicity, 517 


synergies, across trait families, 225 


synonyms, 642 


syntactic structures, deploying, 416 


syntagms, 642 

synthesis tasks, for expert systems, 97 

system(s). See also expert systems 
clear understanding of, 444 


creating to perform at a high level, 444 


diagnosing and troubleshooting of, 96 

improving, 444 

interfering with expertise and reducing performance, 466 
multi-team, 467 

teaching as, 433-434 


System 1 or Type 1 reasoning, as rapid, automatic and unconscious, 333 
System 1 pattern-matching processes, 460 


System 2 errors, as infrequent and unexpected, 334 


system complexity, effects on individuals, 719 


system components, knowledge of, 720 


system control, gradual shift to learner control, 806 


“system” errors, 332 


system experts 


in the context of co-production of knowledge, 141 
residents as, 138 
system interface, 719 


system knowledge, differing, 138 


“system of playing methods,” in chess, 604 


“the system of professions,” competition within, 129 


system of teaching, improving, 444 


system performance, enhancing, 188 


system state, 180 


systematic activity, of musicians, 538 


table tennis, practicing thousands of hours, 44 
table tennis players, gaze of expert, 685 
tacit knowledge, 772-774 

acquiring, 27, 774, 781-783 


action-oriented and procedural in nature, 773 
articulating during in situ instruction, 785 


assessing, 776 


associated with success, 778 


association with expert performance, 778 
defined, 772 

developing, 780—784 

development and application of, 771 


of engineers, 458 


enhancing, 780 


expert identified and measured, 777 

of experts, 455—456 

as explicable, 85 

exposure to, 27 

facilitating expert occupational performance, 774 
facilitating expert performance in non-work domains, 774 
facilitating performance, 784 

features of, 456 

functional role of, 773, 784 


future research on, 784 


going beyond procedural knowledge, 773 


importance to expertise in wisdom and in teaching for wisdom, 779 


level of professional development and, 778 
making explicit, 780-781 


for management, 778 


methods for uncovering, 12 


methods of making explicit, 780 


not an automatic response, 773 


practical intelligence and, 770 


preceding explicit knowledge, 774 


rising to awareness while performing a task, 784 
role in practically intelligent behavior, 774 


significant gains in for the Condition-and-Action method, 782 


similar to job knowledge, 773 


similarities with procedural knowledge, 773 


weak association with personality, 776 
weakly correlated with general intelligence, 775 
Tacit Knowledge for Military Leadership inventory (TKML), 778 


tacit knowledge inventory for auditing, 778 
Tacit Knowledge Inventory for Managers (TKIM), 778 


tacit knowledge scores, regarding safe driving, 779 


tacit-knowledge inventories, 776 


tactical combinations, embedded in a game position, 606 
tactical skills, 275 
tactile expertise, 237 


tactile sensitivity, 552 
“take-the-first” heuristic, 603 


talent 


development program, 285 
identification, 265, 278, 667 


potential mechanisms for explaining, 610 


as a prerequisite in many domains, 143 


requiring minimal external stimulation, 836 


search and development, 665—668 


Taliesin, with an artificial waterfall, 825 


tangibles, for creating a new venture, 398 


target, responding in a group of stimuli, 815 


“target condition,” establishing, 444 


target shape, finding within a more complex set of lines, 579 
task(s) 
activities elicited by standardized tasks, 198 


analysis of, 197 

appropriate for persons with specific levels of expertise, 802 
classes of, 96 

completing, 343, 511, 748 


demands of, 18 


diagram of, 187 


distractions, 728 


domains, 62 


environment of, 64, 719 


expertise involving representations of, 69—70 
goals of, 157 

knowledge, 459 

in knowledge-based systems, 85 


management strategies, 724 
measuring the fixed capacity of STM dramatically improved by training, 
698 


mental models, 513 


prioritization, 730 


related to target performance, 398 
representative, 14, 70 


requiring longer durations of recall, 198 


saturation, 728 
structure of, 806 


of a writer poorly structured, 413 


task performance 
changing as a result of thinking aloud, 196 
initial, 217 
methods of generating models for, 197 
task practice, 218, 222 


“task specificity,” of expertise, 67 


task-relevant knowledge, capabilities of, 62 


task-relevant training, correlation with performance, 538 


task-specific knowledge, 89 


task-specific methods, 86 
taste 


components of, 238 


identifying, 238 


tax advisor, accompanying a tax preparation program, 98 
taxi drivers. See also cab-drivers 
brain effects, 245 


finding a particular destination, 244 


licensed practiced for 35 hours every week, 245 


more experienced generating a larger number of possible routes, 66 


studies of expertise, 402 
taxonomic knowledge, representing, 99 
taxonomy, of teacher knowledge, 441 
teacher effectiveness, measures of, 438 


teacher-learning programs, 448 


teacher-level variance, in student learning outcomes, 439 


teachers. See also expert teachers 
able to integrate the skills of teaching with concepts and knowledge, 442 


as actors with a specific agenda, 432 


allowing to monitor and provide feedback on thinking, 207 


American, helping to increase student discussion in mathematics lessons, 
439 


analyses of classroom video clips, 448 


assessing a given individual’s current performance, 763 
assessing current level of performance, 755 
assigning individualized practice tasks with immediate feedback, 756 


becoming experts, 443-448 


coming and going, 444 


creating learning opportunities for students, 440 
designing practice activities, 755 


education of, improving teaching, 448 


effect on student outcomes, 440 


effects on students’ learning using random assignment, 438 
experienced shifting attention among multiple views, 443 


as expert in one environment but not in another, 434 


expertise connecting to student outcomes, 437 


getting students engaged with studying content, 432 
having knowledge, skill, and judgment, 448 


improving the expertise of, 444 


managing content and students, 432 


meeting regularly in groups, 446 


monitoring the students’ attained practice goals, 756 


not seeing their own skill as needing to be improved, 445 


producing strongest gains on achievement tests, 432 


producing student gains on standardized state tests, 438 
professional development, 448 


requiring the cooperation of students, 431 


responsive to individual students and attentive to the class as a whole, 443 
skills and standards, 65 


societal expectations for, 109 


support by exceptional, 13 


versus teaching as the focus of improvement, 445 
testing changes in routines, 448 
time spent in performance, 445 


viewing video clips of authentic classroom episodes, 441 


walking through example problems, 434 


teaching 
analysis as the key to developing expertise, 447-448 


as a complex socio-cultural system, 448 


as a complex system of interacting elements, 431 


constrained by a number of variables, 433 

as contextual, 438 

creating conditions for deliberate practice, 445—446 
as cultural activity, 434—435 

definition of, 432-433 


differences across countries, 434 


enacted as knowledge dissemination, 36 
for ethics, 780 


expert, as a highly contextualized endeavor, 440 


expertise and expert performance in, 431—448 


expertise existing within a cultural matrix, 439 


expertise in, 436—439 


factors co-determining the results of, 434 
helping students achieve the learning goals valued by society, 433 


improving in two distinct ways, 444 


including planning and reflection before and after the lesson, 433 


initial approaches, 338 
involving adaptive expertise, 436 


methods based on demonstration and imitation, 558 


more like dinnertime conversation than flying an airplane, 435 
more like driving to work than like shooting a rocket to the moon, 445 
nature of, 432-433 


not an individual endeavor, 431 


often observed only by students, 436 


practicing the skills of, 442 
presenting a challenge for models of expertise, 448 


requiring constant adjustment, 438 


researching the effects of on learning, 440 


of science, 33 
structure of, 444 
as a system, 433-434 


ubiquity of, 432 


in the United States compared with teaching in high achieving countries, 
434 


as “working in relationships,” 432 


teaching routines 
aim of improving, 448 


implementing non-optimal, 436 


teaching strategies 


based on an assumption of a general skill as ineffective, 337 


generation of alternative, 447 
team(s). See also expert teams; virtual teams 


activities of, 508 


aspects necessary for ensuring high performance, 522 


cohesion of, 520 

commitment of, 518 

communicating effectively as critical, 723 
described, 507 

deviations of novice considered as errors, 510 
effectiveness of, 508-509 


expert performance of, 747—748 
of experts, 506, 507 


goals of, 509, 522 
helping behavior, 510 
identity of, 518 


innovation as a function of team composition, 516 


interviews, 168 
involving remote collaboration, 458 
literature of, 507 


mediators, 510 


new ventures founded by, 391 


performance of, 508 


scaffolds as coordinating mechanisms in temporary groups, 510 


working in a natural situation, 461 


team adaptability 
defined, 510 
described, 521 


as the driving factor of team adaptation, 510 


team adaptation, 510—511 
conceptualizing as a process, 521 


critical for understanding how expert teams operate, 510 


enabling team members to acquire expertise through learning, 511 


literature of, 521 
model of, 510 


as a nomological network, 510 


as an outcome, 521 


as a process, 511, 521 


promoting successful, 511 


role of as a mediator between team learning and team performance, 511 


team affect research, 508 


“team based” skills, 506 


team behaviors, cognition, and affective states, 509 


team cognition. See also cognition 


current work in, 403 


encompassing a collective team awareness, 508 
models for, 168 
studies on, 458 


team composition, 515—51 


aspects of, 521 
described, 515 
having different definitions and conceptualizations, 521 


literature of, 515 


maximizing team processes and performance outcomes for space crew 
teams, 516 
as a predictor of team creativity, 516 


of two different types of thinking styles, 515 


team coordination 
decreased with virtuality, 518 


training and measuring, 465 


team design processes, 378 


team leaders 
clarifying team member roles, 174 


evidencing skill in identifying key causes, 297 
team leadership, 514—515. See also leadership; shared leadership 
defined, 514 


new avenues for researching, 509 


setting apart an expert team from other teams, 514 


uncovering the underpinnings of, 514 


team members 
engaging in learning processes via evaluating past instances of 
performance, 511 
knowledge, 459 


lacking a shared visual field and access to non-verbal cues, 459 


letting others in on their reasoning, 459 


preference for working on multiple tasks at once, 513 


relying on shared team knowledge, 508 


technology as, 465 
team mental models, more related to team processes, 513 


team model, for knowledge elicitation, 168 


team or organizational unit, knowledge elicitation challenging for, 168 


team practice 
best discriminator after the age of 12 years, 656 


effect on performance, 758 


team process and function, experts at, 507 


team sports 
coaches compensating for a weakness in one player, 666 
identifying game situations where a given player needs to make a quick 


decision, 705 


patterns formed by the relative locations of players, 681 


requiring athletes to make fast decisions and execute rapid responses, 206 


team tacit knowledge, measures of, 776 


team training, 522 


team virtuality, 516, 521 


team-centered communication, mitigating errors through, 459 


team-level rewards, implementation of, 522 


teamwork 


features indicating the existence of expert, 519 


indicators often unseen or unmeasured, 519 


measuring physical, proxemics, and kinetic indicators of, 519 


teams adept at, 506 


technical and tactical demands, of a sport, 667 
technical knowledge, weak correlations with tacit knowledge, 774 


technical performance, cases of, 300-301 


technical practice, undertaken to master the piece, 539 


technical procedures, errors and successes for surgeons, 346 
technical skills 


acquired on low-fidelity bench models, 341 


among different populations of elite youth soccer players, 277 


as a component of medical practice, 340 
objective measurement of, 340-341 
techno-centric development process, 172 


techno-centric mindset, 174-175 


technocracy, critique of, 22 


technological solutions, tendency to turn to, 188 
technology 


becoming ubiquitous in NDM environments, 466 


expert performance differing based on type of, 366-367 
NDM and, 465 
training for new, 367 

technology design, for NDM, 466—467 


technology displays, improving experienced pilots’ situation awareness, 367 


technology-centric design disciplines, ignoring human cognition, 171 


technology-push design strategy, driven by a political agenda, 172 


TED system, for the Army’s M1 Abrams tank turbine engine, 96 


telegrapher, ten years of experience required to become a professional, 11 
telegraphy, performance plateaus, 194, 752 
telephone help desks, 96 


television, language used on, 646 


television programs and movies, vocabulary required to watch, 637 
template theory 


direct implementation of, 605 


explaining results found on blindfold chess, 606 


showing high-level, schematic structures (templates) evolving from 


perceptual chunks, 600 


temporal accuracy, maximal, 566 


temporal discrimination thresholds, 567 


temporal occlusion, 247, 661, 679 
temporal plane, 559 


temporal precision, 562 


temporal relationship, between key elements, 684 
temporal TMS, 513 


10,000 hours of practice, required for expertise in violinists, 625 


“10-year rule,” 425-426. See also decade of practice 


holding for creative domains, 820 


for musicians and composers, 540 
qualifications and complications regarding, 318 


seeming to hold for professional writers, 425 


tennis 
anticipation activated areas responsible for perception of body movements, 
246 


anticipation task, simulation based, 664 


more activation in experts’? AON when predicting where different shots 


would land, 247 
an opponent’s serve often exceeding 130 mph, 660 
think-aloud protocols during the changing of sides, 206 


tennis players 


anticipating what the opponent will do next, 660 


anticipating where the ball is going to land, 234 


performance not improving after decades of weekly playing, 752 


tension view, arguing that expertise works against creativity, 813 
tensor based morphometry (TBM), 559 

Tentative Ask, 400 

Terminal Radar Approach Control task, 219 


test 
of running speed, 214 


yielding reliable results, 214 


testing 


paradigms, 589 


as the tail wagging the dog, 9 
test-retest method, 214 
test-retest procedures, 214 
test-retest reliability, 263 


Texas Hold’em (poker), playing against a computer, 203 
text(s) 
comprehension as a linear relationship with percentage of vocabulary 
known, 635 


professional revision of, 423 


reading a wide range of authentic, 637 


relating to what a reader already knows and stimulating new thinking, 416 


reviewing, 414 


understood by a second language user knowing the most frequent 2,000 
words only, 638 
text production 


emotional and motivational factors involved in, 420 


not occurring in a linear sequence, 415 
processes of, 414 


text representation, developing, 423 


text-based simulations, 338 
Thales, 134 
Theo (British Springer Spaniel), 50 


theorems, proving, 60, 87, 89 


theoretical accounts, of expert performance in older age, 839-840 


theoretical concepts, describing the two sets of respective preconditions, 106 


theory of evolution, potential to organize and unify the social sciences, 40 
thesis, publicly defending, 6 


“think ahead of the aircraft,” during the flight, 726 
“think aloud.” See also thinking aloud 


method of instructing participants to, 62 
subjects asked to, 194 
“think aloud” methodology, 62, 207 


think-aloud protocols 
cognitive processes associated with more experience, 204 
collecting, 72 


demonstrating the intermediate effect, 803 
describing and diagnosing thinking outside of traditional laboratory 
studies, 207 


with expert entrepreneurs revealing reasoning heuristics, 394 


of a good club player and a chess expert, 202 


identifying chess moves, 702 


investigating experts’ problem-solving, 112 


with mental calculators, 704 


skilled chess players evaluating consequences of mentally moving chess 
pieces, 701 


studying a representative sample of expert entrepreneurs, 392 


tracking participants’ thought processes on motor tasks, 803 
think-aloud verbalizations, 14, 206 
thinking 


addressing, 61 

introspective analyses of, 193 
Thinking, Fast and Slow (Kahneman), 85 
thinking aloud. See also “think aloud” 


common alternative to, 205 


giving overt expression to sub-vocal verbalizations, 194 
performance of versus individuals who completed the same tasks silently, 
196 
when choosing the next chess move, 602 
thinking outside of the box, 832 


thinking styles, in a team, 515 
thinking time, blunders from decreasing, 603 
third age (until age 70), 849 
Third International Mathematics and Science Study (TIMSS) video studies, 
434, 439 
Third International Maths and Science Survey (TIMSS), 627 
thought(s) 
basis for, 35 
capturing expert with protocol analysis, 192—207 


giving verbal expression, 196 


reoccurring with the same stimulus, 193 


reporting in sequence, 198 


technical problems in articulating theories and models of, 60 
verbal expression of, 193, 198 


thought processes, assessing during performance, 198 


threat relations, identifying between chess pieces, 242 


Three Dimensions of Expertise, 28 

three-dimensional form and space, conveying the illusion of, 584 
tic-tac-toe, no such thing as expert performance, 45 

time 


classroom teaching constrained by, 433 


creating by the anticipatory encoding of movement pattern information, 
681-684 


dedicated to practice alone reduced for professional musicians, 761 


time analysis, indicating work-rate, 258 
Time Budgets, Diaries and Analyses of Concurrent Practice Activities 
(Deakin, Coté, and Harvey), 257 


time constraints, 335 


time lags, associated with technology, 516 


time motion analysis, 258 
time motion studies, 296 


time periods, in profiles of elite performers, 265 


time pressure 
minimal impact on accuracy, 347 


within groups impacting complex cognition and behavior, 467 


time structures, analysis of, 554 


time windows, early in event sequences, 679 


time-varying covariates, 281 


timing, of writing, 421 


Timmer, Marianne, 277 


tissue, removed during radical prostatectomy surgery, 346 


TMS. See transcranial magnetic stimulation (TMS) 
“To Err Is Human” (Kohn, Corrigan, and Donaldson), 331—332 


tonality, material structured according to rules of, 541 


tongue, 556 


tools 
becoming extensions of persons, 772 
building testable models, 86 
complexity of increased with the expansion of brain size, 45 
humans using, 772 
including enhanced representational systems, 465 
making more effective, 41 


sharing uncertainty or ambiguity, 465 


use acquired accomplishing a desired goal, 772 
used by professionals, 133 
used by writers, 421 


top-down and bottom-up views, integrating, 587 


top-down approach, designers deviating from, 383 


top-down explanations, 584—586 


top-down goal-driven process, 717 


top-down methods, facilitating appropriate visual selection, 587 


top-down position, closely associated with art historian E. H. Gombrich, 584 
top-down processing 
describing, 580 


operating in tandem with bottom-up processing, 717 


visual expertise engaging, 236 


top-down results, summary of, 586 


top-down strategies, resolving perceptual ambiguities, 587 


topographical skill, of taxi drivers, 244 


tournament chess performance 


captured in a controlled environment, 701 


closely associated with performance on selecting-the-best-move task, 698 


tournament play, effect not statistically significant after taking deliberate 
practice time into account for chess, 608 

Tower of Hanoi, 155, 195 

Tower of London, solving, 607 

TRACON (Terminal Radar Approach Control), complexity of, 218 


trade guilds, in medieval times, 127 


trade-offs, of beginning teachers, 443 
Trail-Making B, 848 


train engineers, specializing on specific geographic areas, 358 


trainability, 52 


trained motor patterns, 558 


trainees 


minimizing the number of washing out of a training program, 223 


presented with realistic scenarios, 460 


reviewing rankings and rationale provided by operational experts, 185 


training. See also individualized training; musical training; perceptual 
training 
ability to anticipate, 687—689 


accuracy of information, 263 


before the age of 7 years resulting in changes in white-matter connectivity, 
962 


in causal analysis, 297 
designed for particular performers by teachers and coaches, 14 


effects of different types of, 281 


effects of on memory for briefly presented chess positions, 698 
for expertise, 459-460 


factors distinguishing experts from non-experts as the result of, 257 
hazard perception, 364—365 


hours per week, 281 


individualizing in college, 757 


intense necessary for modifying joints, 754 


interventions, 782 


late onset of as a triggering factor, 566 


leverage point for as a cognitive one, 368 
logs, 259 
NDM.-based, 459-460 


necessary for musical expertise, 552 


needed to direct workers’ attention to critical information, 175 


needed to reach high levels of performance in any domain, 10 


older adults requiring specific types of, 846 


outcomes improving with expert teams, 522 


outside the current comfort range of the learner, 445 
for pilots as rigorous, 726 
power to mitigate age effects, 842 


practice techniques among chess players, 607 
proposing types of, 77 
teaching novice squash players the cues used by experts, 688 


types of, mediating improved performance for years and decades, 754—757 


using a challenging scenario with several decision points, 185 


years of intense required to become an internationally acclaimed 
performer, 751 
training activities 
other types of, 757—759 


specialized, 75 


training and experience, after a limited period of, 752 


training and practice activities, describing different, 755 


training environments, practicing to attain specific changes, 75 
training group 
engaging in thoughtful analysis, 657 


increased mental effort during practice, 657 


training methods 
for acquiring complex cognitive mechanisms, 77 
in chess, 606 

training programs 


for judges of case studies, 294 


“leveling” individual differences in performance, 217 


training/practice, importance of in skill development, 257 
trait(s) 


associated with learning the most, 216 


associated with particular challenges, 516 
defined, 213 
families of, 221 


operating synergistically, 226 


required for expertise, 45 
trait complexes, 226—228 


facilitative in the development of knowledge about different domains, 229 


at the heart of Ackerman’s investment theory of adult intellectual 


development, 227 
predicting educational and occupational opportunities, 228 


as useful predictors of individual differences in domain knowledge, 227 


trait determinants, of expert level performance, 221 


trait predictors 
of expertise, 221—225 


of initial task performance, 221 


trait quality, 44 

“traits” approach, personality variables researched as part of, 390 
trajectories of developed performance, recording for individuals, 751 
Transactional Ask, 399 


transactive memory 


groups forming, 141 

notion of “shared” related to, 459 
transactive memory systems (TSM), 518 

defined, 512 


evaluated via a self-report measure, 512 


literature on shared cognition, 521 
transcranial magnetic stimulation (TMS), 512-513, 567, 568 


“transdisciplinarity,” 138 


transfer. See also near transfer 
of expertise, 820, 822, 823-828 
local, 821 
low-level, 669 
in older adults, 848 
of training, 228, 460 


transfer learning, work on, 89 


transfer quotient, promoting skill transfer, 668 
transfer-of-expertise analysis, 823 


transformational leadership, at the team level, 520 


transformations, between cognitive states, 178 


translating, sub-process of, 414 
translation, of languages, 704 


transnational sociology, 129 


transparent decision aids, 495 


transparent systems, 84 
transportation 
in the first edition of the Cambridge Handbook, 358 


moving people from one point of this planet to another, 356 


offering complex embeddings of cognitive and non-cognitive factors, 357 
some aspects becoming automatic, 358 


value of experience in, 356 


transportation domain, expertise in, 356-368 


trauma teams, investigation of expert, 510 
treadmills, building, 749 


treatment methods, for musician’s dystonia, 568 


“a tree must bend while it is young,” 563 


trial-and-error techniques, for design modification, 384 
triangulation, confirming case observations, 292 
triathletes, 266 

triggering event, 678 


triggering factors, for MDs, 564 
triple helix, as the first model of DNA, 823 


trumpet players, inhalation and expiration pressures in, 542 


trust 


among members of virtual teams, 518 
role in cross cultural teams, 510 
by team members, 508 
“truth,” 136, 137 
“truth machine,” peer review as, 138 
Turing Test, 27 


TV and movies, watching, 647 


TV shows, pitching on, 399 


two-dimensional array of neurons in M1, coding for three-dimensional 
movement in space, 556 
“Type 1 domains,” establishing a causal connection between proficiency at 


specific activities, 400 


Type 1 processes 
evidence linking to exemplars as strong, 336 


as an impediment to the quest to become an elite performer, 334 


reflecting direct unconscious retrieval of prior exemplars (hypotheses), 336 


relying on retrieval of specific exemplars, 337 
shortcuts or “heuristics” used by, 333 


Type 2 domains, prediction of individual or aggregate human activity, 400 


Type 2 processes 
analytical knowledge consisting of different knowledge representations, 
337 


computational processing in memory, 336 


constraining to increase errors, 335 
relation between errors and, 336 
requiring working memory, 335 


Type 2 (System 2) reasoning, 333 


typicality, sense of, 455 


typing 
advance coordination of fingers in, 837 


attaining acceptable performance, 752 


evidence for plateaus, 752 


increases in speed and, 758 


as procedural knowledge, 773 


typing speed, 224, 758 


typists 
having more gray matter, 248 


increasing typing speed, 758 


older expert, 844 


ubiquitous expertises, 24, 28 
UK National Health Service, 171-172 


unacceptable word sequences, influence of the learner’s L1 on using, 646 


unanticipated contingencies, 90 

unattainability, 762 

uncertain inference, dealing with, 91 

uncertainty 
as a characteristic of entrepreneurship, 400 
dealing with, 457 
as hallmark of NDM domains and situations, 457 


having to do with unknown and even unknowable distributions, 394 


involving, 479 
managing, 456 


principal forms of, 457 
rough estimates of, 91 
successful designers good at coping with, 374 


technology and system design addressing issues of, 467 


unconscious thinking, 107, 814 


undergraduates, reviewing key facts and anomalies, 301 
under/overconfidence, 487 


understanding. See also representative understanding 
as a function of the integration of multiple pieces of data, 721 


more focus on at higher levels of expertise, 726 


of what is perceived, 720 


“unexpected discoveries,” during the design process, 379 


unguided learning, placing higher cognitive demands on novice learners, 798 
unguided problem-solving, vs. worked examples, 797—798 

Union Carbide disaster in Bhopal, 22 

unit structure, 99 

units of analysis, 294, 314 


universal control system, cerebellum serving as, 557 


“universitas magistrorum et scholarium,” 135 


universities 
accumulating and explaining knowledge, 6 
described, 135-136 
leadership simulation, 297, 300 


student demand for entrepreneurship classes in, 391 


University of Missouri, School of Journalism running a daily city newspaper, 
425 


University of Paris, 6 


university teachers, specialised role of, 136 
unobtrusive measures, providing incorrect information about the team’s 
functioning, 519 


“unprincipled” behavior, not equating “opportunistic” with, 384 


“unreasonable” place, world as, 818 


unshared information, 140 
unstructured pattern, in basketball, 682 


unsystematic effort, 168 


upper anterior cingulate gyrus, 240 


U-relation, between age and level of performance, 272 

US intelligence community, following the 9/11 tragedy, 172 
US Internal Revenue Service, recognizing dog expertise, 50 
US Olympic basket ball team, in 1992, 506 


US president, performance of contingent on prior experiences, 315 


usability of knowledge, alternative proposal about, 71 


usability testing of software, 207 
“usable” knowledge for teaching, measure of, 441 
use, knowledge of, 643-644 


U-shaped relation, in data with younger workers, 838 


utility analysis, of judgment accuracy, 453 
“utility of effort,” 225 


vaccines, controversies over, 22 


vacuum pumps, not efficient enough, 829 


valid outcomes, for patients requiring long-term follow-ups, 203 
validity 

concept of, 215 

of data, 263—264 


implicit in experimental research, 229 


of judges’ appraisals, 294 


of procedures, 294 


referring to whether a property measures what it sets out to measure, 215 


of verbalized information while thinking aloud, 197—198 


“value add” models, of student achievement, 432 


value judgments, affected by irrelevant variations, 487 


values, assessing, 486 


1 


variable definitions, in historiometric inquiries, 314-31 


variable environments, placing strains on organisms, 41 
variable type aspect, of team composition, 521 
variables 


accounting for firm survival, 393 


correlated to an individual’s current level of performance, 271 


selecting to include in a longitudinal study, 285 
sets of, for classical music composers, 319 


underlying performance, 275 


variance, partitioning, 217 


variance structure, establishing, 281 


variance truncation, 314 
Vedic priest, 244 


vehicle control skills, 734 


vehicle operators. See also drivers 
adapting to the new age of technology, 367 


experienced more likely to respond to a situation change, 360 


experienced requiring greater training, 367 


gaze patterns of experienced and less-experienced, 360 


learning to self-regulate behavior, 357 


ventral premotor cortex, expert dancers activation of, 246 


ventral tegmental area, 554 


venture Capitalists, 399 
venture formation, 392 
venture investors, 391 
Venturi, Robert, 375 


venturing, success in, 391 
Venturing (name of imaginary product), 406 


venturing instrument, 405—408 


verbal ability, of professional writers, 416—417 


verbal explanations of technical diagrams, 798 


verbal interaction, during communication tasks, 785 


verbal overshadowing, effects of, 197 


verbal protocol analysis, 60 
verbal protocols, masters using larger structures than chunks, 600 


verbal reports 


collected during near and far task decision scenarios, 664 


on export thought processes, 193-195 


generating more detailed, 194 
made by “experts” versus novices, 291 
methodologies, 206, 538 


methods and instructions used to elicit, 195 


monitoring experts’ performance, 12 
procedures, 196 


protocols for probing conscious strategies, 680 


regarding mental calculators, 704 


suggested an increase in higher-order cognitive processes, 662 
of thinking, 193 
verbal working memory, demands on from planning novel ideas, 416 


verbalizations 
corresponding to vocalizing “inner speech,” 195 


of intermediate sums, 197 
levels of, 195 


of one’s experiences, 780 


of thoughts, 196, 207 


verbalized information 


during silent thinking, 195 
validity of, 197-198 


veridical appearance of an object, perceiving, 581 


vernacular (folk) musicians, 536 


vertex types, prevalence in a variety of languages, 157 


vertical lines, perceived as longer than horizontal lines, 581 
VER landings, 361 


vibrating stimuli, worsening musician’s dystonia, 568 


video cameras 
helmet-mounted, 465 
using with NDM, 465 


video clips, in soccer, 662 


video-based training, requiring perceptual judgments, 689 


videos and detailed records, collecting for particular patients, 203 


videotaped lessons, collected from teachers’ classrooms, 438 
videotapes, of eighth-grade mathematics and science lessons, 434 


vigilance and monitoring deficiencies, 728 


vignettes, depicting practical problems, 776 


violin, technical demands of the left fingering hand, 564 
violinists 


accomplished having accumulated more practice, 755 


categorized into groups based on level of attainment, 654 
cramp, 563 


from different degree programs, 537 


early start of training, 536 


forearm rotation of, 542 


left hand more frequently involved, 564 
tending to be practice fanatics, 537 


time practicing alone as the most relevant activity, 755 


“virtual patients,” 332 


virtual reality simulator, conducting missions in, 730 
virtual team leaders, 518 
virtual teams. See also team(s) 


development of swift trust in, 510 


leadership in, 509 


overcoming challenges associated with large team sizes, 517 


performance of, 516, 518 


relying on electronic tools to collaborate and coordinate, 516 


team members being satisfied with their leader, 517 
virtual tools 


communication via, 514 


utilizing in real-time, 517 


virtuality 
associated with enhanced unique information sharing, 517 
effects, 517, 518 


influence of in novel settings, 518 


relevant aspects of for categorization purposes, 521 
of a team, 516—518 
virtuoso piano repertoire, demanding more from the right hand than the left, 
564 
virtuosos, requirements for, 552 
vision, central versis peripheral, 686-687 


“visionaries,” introducing completely new concepts and constructs, 385 


visual acuity, changes in, 151 


visual aids 
as beneficial for certain tasks, 496 
benefits of, 494 


improving skilled decision making, 497 


for many types of risk communications, 496 
promoting risk literacy, 495 
providing, 494 

visual art, 576, 592 


visual artists 


creating depictions, 592 


depicting real or imagined visual scenes, 160 


exploiting many technical devices, 576 


visual association areas, in the brain, 240 
visual attention, 155, 586, 587 
visual binding, 155 


visual categories, 156 


visual coding, 622 

visual cortex, 157 

visual expertise, 236, 248 
visual field, 686 

visual gaze, 664, 684 


visual grouping, impacted by formal syntax, 156 
visual illusions, 581, 583, 589 


visual information, integration of, 556 


visual input, streams of processed in parallel, 577 
visual object recognition, 152 


visual operations, 153 


visual perception. See also perception 


as an aid to developing good number skills, 619 
discipline-specific ways of altering, 154 


evidence regarding low-level or sweeping changes to, 160 


organizing to facilitate identification, 160 


visual perceptual abilities, 619 
“visual pivots,” 661 


visual realism, 577 


visual representations 
binding terms into objects, 156 
modified format with additional, 800 


visual scan patterns, in helicopter overland navigation, 361 


visual scenes, construction of, 154 


visual search, facilitating, 159 


visual selection and decision making, critical for realistic depiction, 585 


visual system, inferring the actual structure of objects and scenes, 581 


visual tasks, implementing, 152 


visual tokens, ideas as, 380 


visual word form area, 156, 157 


visual working memory, 622 


visual-information pick-up, 660-662 


visually accurate rendering, 577 


visually engaged art experts, testing, 589 


visuo-spatial features, perceiving in sketches, 379 


visuo-spatial images, generation of, 241 


visuo-spatial interfering tasks, negatively affecting problem-solving 
performance, 606 
vocabulary 


behaving in terms of frequency, 637 


breadth of, 639 
knowledge of, fundamental to all language use, 634 


mastery of, 640 


mid-frequent, 638 


necessary to use English at the basic end of the proficiency continuum, 638 
as one aspect of comprehension, 637 


vocabulary depth, 639 


as a complex construct, 643 


conceptualized in relation to individual words, 640 
reliable and comprehensive measures of lacking, 648 
understanding, 640 


vocabulary knowledge 
components of, 648 
contributor to success in language performance, 634 
depth of, 639-640 
as a good predictor of general proficiency in a second language, 634 
involving both breadth and depth, 639 
network view of, 641 
Vocabulary Levels Test, 636 
Vocabulary Profile, 636 


vocabulary size 


correlating positively with judgments of writing effectiveness, 417 

larger suggesting a high level of language exposure and learning 
experience, 639 

for LS, 637-639 


Vocabulary.com application, 648 


vocational interest themes, 225 
voxel based morphometry (VBM), 235, 559 


Wagner, Richard K., 772 
waiters 


exceptional memory for dinner orders, 200 


keeping in mind precise orders for up to 20 people, 623 
The Waste Land (Eliot), 425 
water, not compressible, 829 
Watson, John B., 193, 194 
ways of being, 36, 37 


“we know more than we can tell,” explaining why, 455 


“weak and strong methods,” in AI, 61 


weak areas alone, practice in, 807 


weak executive functioning, 816 


weak methods, 61 
weakly activated ideas, becoming sensitive to, 816 
weather conditions, skilled performer adjusting performance to changed, 73 


weather forecasters, 183, 722 


weather map comprehension, visual salience and, 153 


Web, technology for traversing, 93 
Weber, Max, 137 


weekly training activities, increasing or changing, 754 


Weighing pros/cons of alternatives, in RAWFS, 457 
weight, associated with a rule, 91 

weight classes, dogs sorted into, 51 

weight-pulling, sport of, 51 

Welsh laws of Hywel Dda, 50 

Wernicke’s area, 415, 554 

“Western art music tradition,” 536 

the “what,” of The Ask, 398 


“what-if” type thinking, 726 


white matter, differing between different instrumentalists, 561 


white matter density, increasing as a consequence of musical training, 553 


white matter microstructures, 559 


white matter tracts, 560 

white-matter organization, in early- and late-trained musicians, 561 
the “who,” of The Ask differing, 398 

Who is Musical? (Billroth), 535 


whole-case approaches, 338 


Wikipedia entries, employed in case studies, 293 
wildland firefighters, 456 

Wilkins, A. F., invention of radar, 828—829 
William of Ockham, 135 

Williams sisters, in tennis, 625 

Wiltshire, Stephen, 579 


wine expertise, 258 


wine tasters, 238 


“wing warping,” 827, 828 


wisdom, 779 


within-person processes, allowing for the investigation of, 404 


women, displaying a lower chess-specific self-esteem, 608 
Woods, Frederick, 312 
woodwind players, both hands equally likely to be affected by dystonia, 564 


word(s) 


ability to use strongly related, 641 


following formulaic patterns, 643 


interconnections between, 641 


knowing involving the mastery of nine aspects, 640 


learning many in a limited amount of time, 640 
small number of occurring very frequently, 637 


word acquisition, proposed stages of, 640 


word association knowledge, 642 


word choice, diversity in, 417 
word combinations, learning, 635 
word families, 634, 637 


word forms, processing regions for, 157 


word images, sequences of, 414 
word knowledge 
components of, 641-642, 648 


conceptualization of, 640 


dimensions of, 639 
word level, of a text, 415 


word meaning, knowing the boundaries of, 642 


word pairs, 219 
word parts, 640, 642 


wordlists, memorizing, 640 


work. See also professional work 
anthropological studies of, 166 


changing nature of, 100 
defined and cognitively steered by specific interpersonal work settings, 
132 


expert, as an activation of expertise in context, 141 


institutionalization of expert, 143 
reallocating into more efficient work packages, 174 


supporting, 172 


work activities 


aligning with learning processes, 113 


compared to deliberate practice, 424 


development of occupational expertise through, 113-116 


engaging in novel generative of new learning, 114 


learning and, 114—115 
providing access to knowledge, 118 


routine providing learning experiences, 115 


work design documents, employed in case studies, 293 
Work Domain Analysis, 175 


work environments 


engineering, 420 


predominated by non-routine tasks, 781 


work experiences, supporting the development of occupational expertise, 
416-121 


work narrative, mapped onto a Decision Ladder, 179 


work plans, as a basis for analysis, 302 


work products 
analysis of, 302 


employed in case studies, 293 


observations of, 304 


using as a basis for case studies, 302 
work schedule, adhering to, 420 


work sessions, length of, 422 


work settings 


affording occupational experiences, 108 


objective performance criteria in, 167 


work strategies, appraised for scientists, 301 
Work Task Analysis, 178-181 


work task trajectories, 180 
work tasks, 178 


worked example effect, 798 


worked example-problem pairs, 807 
worked examples 
vs. unguided problem-solving, 797—798, 801 


workers 


coming to perceive their learning as being unappreciated, 122 


defining expertise on the basis of abilities, capacities or skills, 167 


at different levels of experience following different trajectories, 180 
effective, 166 


engagement in both routine and non-routine problem-solving tasks as 
generative of new learning, 114 

experienced, 166, 176 

identifying as experts, 167 


motivating to engage in deliberate practice, 175 


professional not needing supervisors, 132 
relying on a mix of formal, informal, and affective cognitive processes, 
166 


workers/employees, certification of, 132 


working animals, sharing similarities with human experts, 54 


working memory, 620—624. See also short-term memory (STM) 
ability not predicting SA scores in experienced pilots, 725 


accessing knowledge structures in long-term memory, 797 


activating explicit knowledge, 803 
age-related changes, 838 
assessments of, 493 

as a bottleneck, 717 


capacity and retrieval, 719 


composing placing severe demands on, 417 


cues in for retrieving information in long-term episodic memory, 622 


determining which knowledge held in long-term memory is used to 
determine action, 797 
extensively studied in other domains, 706 


individual differences in the measures of, 707 


influence of on sight reading performance, 707 
limitations of, 414, 805 


limitations on calculation, 620 
limited capacity of, 622, 796 
load, 155, 793 


during mental calculation, 703—704 


not a limiting factor in creative decision making, 706 


novices severely hampered by, 719 


in other domains of expertise, 706—707 


positive relationship with Level 3SA, 725 


representation of game situations, 706 


research demonstrating expanded during addition of numbers, 704 

role on SA, 725 

“sharply limited processing capacity” of used only by Type 2 processes, 
334 


from short term to long term, 70—71 


U1 


during simultaneous translation, 704—705 
sports and, 705-706 
storage, 71, 709 


superior in experts, 696—709 


superior performance supported by expanded, 699 


support of chess playing, 703 


Type 2 reasoning loading on, 333 


types of used in calculation, 622 


workload 
creating a significant challenge for SA, 724 
decreased dramatically despite the reduction in staff, 174 


problems faced by inexperienced pilots, 730 


sharing, increase in, 510 


workplace 


challenges of observation, 168 


as the common turf for concurring occupations, 133 


contributions to the development of occupational expertise, 109 


mathematical skills in, 616 


organization of experiences, 121 


work-related learning, 121 
world 
in flux, 820 
as an “unreasonable” place, 820 
World Chess Federation, rating list published by, 609 
“world” in NDM, 454 
World Memory Championship (WMC), 700, 841 


world records, improvements in many types of sports, 274 


“a world without work,” speculation about, 100 


world-class chess players. See also expert chess players 
able to virtually reproduce all the pieces in the position, 697 


discovering new moves superior to all the previously generated ones, 201 


engaged in systematic search and planning, 702 


recall performance by, 68 


verbalized many good first moves, 201 
world-class creative performance, age function of, 320 
world-class expertise, tending to emerge from a distinctive family 
background, 317 


world-class grandmasters, comparing the quality of play of in standard games 


and rapid games, 603 


“Wranglers,” students of mathematics at Cambridge University, 618 


wrestlers, 655 
Wright, Frank Lloyd, 302, 825-826 
Wright brothers 


control system, 827—828 


development of powered flight, 300 


introduced the principle of balance, 300 
wrist flexors, inhibition of, 567 
writers. See also professional writers; various types of writers 


best work emerging from flow states of absorption in the task, 421 


career, 413 

developing habitual ways of approaching work, 421 
experienced generated twice as many words per burst, 416 
of fiction, 425, 426 


in flow focusing intensely on the task and loosening self-awareness, 421 


forcing themselves to produce line after line, 421 


looking like very busy switchboard operators, 417 


needing coaching through feedback, 424 


problems of, 414 


producing a creative story with a prompt of three words, 416 


restructuring ideas stored in long-term memory, 423 


scheduling only a few hours per day for composing, 424 


specializing in specific rhetorical context, 419 


stages of development for, 422-424 
writer’s block, 420, 421, 422 


writing 


belief in innate talent, 420 


cognitive demands of, 414—416 


development starting in early childhood, 422 


emotional demands of constraining a writer, 420 


evidence of practice in, 424 
extended texts for publication, 413 
fundamental, 414-415 


of graduate students illustrating knowledge transforming, 423 


as ill-structured and types of texts generated by professionals as varied, 
413 
intensive courses, 425 
kinds of, 419 
reluctant sessions, 421 
skills, 425 
writing expertise, 416—422 
t 


as domain dependent, 419 


written characters, transcription of words into, 414 


written composition, psychological model of, 414 


written fluency, not catching up with speech until around the age of 12, 423 


written form, of a word, 640 


written languages, 157 
written method, using the mental analogue of, 621 


written production strategy, 423 


written specialty examination, resulting in a lower patient mortality, 345 


X-chromosome, genes located on, 564 
X-rays 
diagnosing, 70 


learning to read, 344 


from previous patients with known outcomes, 204 


years in practice, as a surrogate for expertise, 345 


years of experience 
NDM studies relying on, 454 


not conclusively predicting expertise, 167 


young and older expert musicians, brain activity of, 841 


Young Decision Making Competency assessment (Y-DMC), 486 


youth athletes. See also athletes 


performance improving with age, 272 


selecting for the youth selection team, 281 


study designs focusing on the complicated development of, 285 


youth competitions, older athletes usually outperforming younger, 272 


youth selections, for sports, 275 


zeal, common characteristic for prodigies, 629 
“Zeigarmik Effect,” 623 


zero-sum game, 433 


“zone of proximal development,” 398 


Z-scores, analyses with, 279 


